linux-ext4 - Re: What is status of metadata

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140117205819.GM9229@birch.djwong.org>
Date:	Fri, 17 Jan 2014 12:58:19 -0800
From:	"Darrick J. Wong" <darrick.wong@...cle.com>
To:	Oleksij Rempel <linux@...pel-privat.de>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: What is status of metadata_csum

On Fri, Jan 17, 2014 at 09:13:11PM +0100, Oleksij Rempel wrote:
> Am 17.01.2014 20:21, schrieb Darrick J. Wong:
> > On Fri, Jan 17, 2014 at 11:39:07AM +0100, Oleksij Rempel wrote:
> >> Am 17.01.2014 09:18, schrieb Oleksij Rempel:
> >>> Am 17.01.2014 03:00, schrieb Darrick J. Wong:
> >>>> On Thu, Jan 16, 2014 at 11:57:20AM +0100, Oleksij Rempel wrote:
> >>>>> What is metadata_csum status?
> >>>>>
> >>>>> i see there are benchmark updates on wiki page, but no updates for
> >>>>> e2fsprogs. 1.43-WIP seems to be really old.
> >>>>
> >>>> Still waiting for 1.43.  I don't know if there are early adopters running
> >>>> 1.43-WIP without complaint, or if simply nobody's using it at all. :/
> >>>
> >>> I didn't tested current master.git - my bad. It is not outdated :)
> >>>
> >>> Beside, the wiki should be corrected. If you enable 64bit, it will need
> >>> extents too.
> >>>
> >>> Are there any existing tools for error injection for file systems?
> >>> Random read error from libata or silent data corruptions?
> >>>
> >>
> >> I was playing with metadata_csum trying to kill my system and now i have
> >> some questions:
> >>
> >> first the test:
> >> #find block number
> >> sudo ./debugfs/debugfs -R "imap <3991825>" /dev/sdf1
> >>
> >> #copy block
> >> sudo dd if=/dev/sdf1 bs=4K count=1 skip=15958163 of=corrupt_block.bin
> >>
> >> #change one byte
> >> ghex corrupt_block.bin
> >>
> >> #kill inode
> >> sudo dd of=/dev/sdf1 bs=4K count=1 seek=15958163 if=corrupt_block.bin
> >>
> >> sudo ./debugfs/debugfs -R "stat <3991825>" /dev/sdf1
> >> #stat: Inode checksum does not match inode while reading inode 3991825
> >>
> >> so, inode is corrupt:
> >> - ext4 can detect inode corruption. Great work! I like it :)
> >> - in this case file will not be shown by Nautilus.
> >> - ls will show some thing like this:
> >>
> >> 4130898 drwx------  8 oleksij oleksij      4096 Jan 17 09:31 Bilder
> >>       ? -?????????  ? ?       ?               ?            ?
> >> ich-einfach_unverbesserlich_2.mkv
> >> 3991826 -rw-rw-r--  1 oleksij oleksij 622038707 Jan 17 11:12 igor.mkv
> >> 3989506 drwxr-xr-x 26 oleksij oleksij      4096 Okt  9 23:11 linux
> >>
> >> it means, i know there is some thing wrong, but i do not know which inode
> > 
> > Unfortunately, either the whole stat call succeeds or it doesn't.  Your best
> > bet to find the inode is:
> > debugfs -n -R 'stat /path/within/fs' /dev/sdXXX
> 
> Ok, looks like user space apps should handle errors in different whey. I
> mean some thing like, show notification about probable FS errors and so on.

AIX has ECORRUPT; maybe we should define that for the kernel/glibc?

> >> - If fs is not fixed, file is lost? are blocks still allocated to this
> >> inode?
> > 
> > Yes, the blocks are still allocated.
> > 
> >> - debufs will not allow me to edit inode corrupt inode.
> > 
> > debugfs -n (disables checksum verification)
> > 
> >> - If i run fsck -y file is lost. Most user wont be able to recover it.
> > 
> > If you answer 'n' to the first prompt about the checksum failure, e2fsck will
> > check the rest of the inode.  If no other problems are found, it'll ask to
> > correct the checksum.
> > 
> > I've wondered if perhaps e2fsck ought to do something like this when there's a
> > checksum problem:
> > 
> > 1. If the user didn't prohibit clearing the inode as the first potential
> >    action, ask if e2fsck should just clear the inode.
> > 2. If not, run the regular checks.
> > 3. If there's still a checksum error, ask to fix the checksum.
> > 4. If the user doesn't want to fix it, ask to clear the inode.
> 
> sounds good.
> 
> > iow, e2fsck --verify-checksums-last /dev/sdXX
> 
> this mean?

It would be a new flag to change the fsck behavior.

> PS: More testing results from today. GRUB fails to install if
> metadata_csum enabled. Should it be reported to GRUB devs?

Yes, please report the bug to them.  metadata_csum is a ROCOMPAT feature, which
means that grub will need to learn about checksums if it wants to do anything
other than rewrites of existing file blocks.

--D
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html