linux-ext4 - Re: e2fsck -D lead to severely damaged filesystem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20180116090352.lsz4mhpfho3noumx@quack2.suse.cz>
Date:   Tue, 16 Jan 2018 10:03:52 +0100
From:   Jan Kara <jack@...e.cz>
To:     Nikola Ciprich <nikola.ciprich@...uxbox.cz>
Cc:     linux-ext4@...r.kernel.org, nik@...uxbox.cz
Subject: Re: e2fsck -D lead to severely damaged filesystem

Hello,

On Mon 15-01-18 08:23:29, Nikola Ciprich wrote:
> we were dealing with slow access to directories with lots of
> files (large maildirs), so after some  tests, I came to conclusion
> that optimizing directories using e2fsck -D (on unmounted FS of course)
> helps a lot. So after testing this on our test box, I did it on production
> mailserver mail volume. The I decided to do some tests on newer kernel,
> so I rebooted test box and got lots of fs errors..
> 
> I checked production box, and it got bad as well:
> 
> lots  of dx_probe:829: inode #15949784: block 35579: comm deliver: Directory hole found
> messages.. 
> 
> 
> so I unmounted fs again, run fsck, and got zillion of:
> 
> Inode 18378187 ref count is 2, should be 1.  Fix? yes
> 
> Unattached inode 18378194
> Connect to /lost+found? yes
> 
> messages.. 
> 
> 
> after ~3 hours, I gave up, and recovered FS from backup.. checking fs after
> "repair" showed that some of large mailboxes vanished completely (and appeared in lost+found)
> 
> I think I can rule out hardware problem, since it appeared on two completely different
> systems after some action.. but I'll try to prepare new test environment and reproduce it.
> 
> What I think might be my big mistake is that I was using quite old e2fsprogs - 1.42.6,
> kernel was 4.4.52 (which I know is also a bit old, we're already testig 4.14.x)
> 
> My question is, was that  some known e2fsck problem which got fixed in new version?

Commit 19961cd000 "e2fsck: fix e2fsck -fD directory truncation" sounds like
fixing a similar problem you've observed. So there's reasonable chance
newer e2fsprogs will handle the filesystem fine. But if not, please do
"e2image -r <device> - | xz -c >ext4.image" *before* running e2fsck -D and
put it somewhere for download. That way we can experiment with the metadata
image and see what exactly does e2fsck do wrong. Thanks!

								Honza

-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR