linux-ext4 - Re: unexpected sync delays in dpkg for small pre-allocated files on ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160601141218.GE5357@thunk.org>
Date:	Wed, 1 Jun 2016 10:12:18 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Gernot Hillier <gernot.hillier@...mens.com>
Cc:	Dave Chinner <david@...morbit.com>, linux-scsi@...r.kernel.org,
	MPT-FusionLinux.pdl@...adcom.com, linux-ext4@...r.kernel.org,
	sathya.prakash@...adcom.com, chaitra.basappa@...adcom.com,
	suganath-prabu.subramani@...adcom.com
Subject: Re: unexpected sync delays in dpkg for small pre-allocated files on
 ext4

On Wed, Jun 01, 2016 at 03:17:14PM +0200, Gernot Hillier wrote:
> I repeated the discussed tests and found comparable results on this machine:
> 
> - 3 seconds dpkg install time on ext3 vs. 80 seconds for ext4
>   on same partition for same package
> - 40 ms for fallocate+write+sync_file_range for writing a few bytes
> - 15 ms for write+fdatasync vs. 45 ms for BLKZEROOUT on raw device
> 
> So this seems to be not bound to one specific disk+controller setup, but
> it still can't be a common problem affecting many people as then we
> would see more reports about it...

OK, so let's try to get common baseline, shall we?  I'm using as my
test package:

http://debug.mirrors.debian.org/debian-debug/pool/main/e/e2fsprogs/e2fsprogs-dbgsym_1.43-2_amd64.deb

% ls -s /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb
1108 /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb
% md5sum /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb
148ee631f46ed2e05a17faf36cc9d630  /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb

# dpkg --purge e2fsprogs-dbgsym
(Reading database ... 322967 files and directories currently installed.)
Removing e2fsprogs-dbgsym (1.43-3) ...

# /usr/bin/time dpkg --force-depends -i /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb 
Selecting previously unselected package e2fsprogs-dbgsym.
(Reading database ... 322947 files and directories currently installed.)
Preparing to unpack .../e2fsprogs-dbgsym_1.43-3_amd64.deb ...
Unpacking e2fsprogs-dbgsym (1.43-3) ...
Setting up e2fsprogs-dbgsym (1.43-3) ...
0.21user 0.19system 0:00.54elapsed 74%CPU (0avgtext+0avgdata 73188maxresident)k
0inputs+10168outputs (0major+24935minor)pagefaults 0swaps

Something else would be useful is to make sure /tmp is a tmpfs mounted
file system (so you don't get any I/O's caused by /tmp), and then do:

# dpkg --purge e2fsprogs-dbgsym
# btrace /dev/sda3 > /tmp/btrace.out &
# /usr/bin/time dpkg --force-depends -i /tmp/e2fsprogs-dbgsym_1.43-3_amd64.deb
# kill %btrace

Replace /dev/sda3 with the device name of your root partition, and
then send us the compressed btrace.out file, which might give us a
clue about what is going on.

Cheers,

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html