linux-ext4 - Re: HUGE slowdown when doing dpkg with ext4 over nbd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20161207162422.GA11436@infradead.org>
Date:   Wed, 7 Dec 2016 08:24:22 -0800
From:   Christoph Hellwig <hch@...radead.org>
To:     Renaud Mariana <rmariana@...ine.net>
Cc:     Andreas Dilger <adilger@...ger.ca>, linux-ext4@...r.kernel.org
Subject: Re: HUGE slowdown when doing dpkg with ext4 over nbd

We had a pretty similar issue lately with SCSI, and I suspect it's
something similar.  There are two interrelated issues:

 - dpkg uses fallocate where it absolutely shouldn't - it creates
   new files with typically a single write call, so fallocate doesn't
   help anything, but actively hurts.
 - For small enough regions ext4 does not create unwritten extents
   but zeroes data on disk, and whenever zeroing out the data is
   expensive this really shows up.  The SCSI case was a device
   that has a horrible slow WRITE_SAME implementation, but for nbd
   we'll just write the zero page repeatedly.

Patching out the stupid fallocate in dpkg will speed ext4 up a lot
(especially for small files) and will probably speed XFS up a tiny
bit as well.  But all my mails to the dpkg folks were simply ignored
unfortunately.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html