linux-ext4 - Re: Proposal draft for data checksumming for ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <05B7E436-A365-41C5-91F6-07E3B3B59B53@dilger.ca>
Date:	Mon, 28 Apr 2014 14:16:27 -0600
From:	Andreas Dilger <adilger@...ger.ca>
To:	Dmitry Monakhov <dmonakhov@...nvz.org>
Cc:	"Darrick J. Wong" <darrick.wong@...cle.com>,
	Lukáš Czerner <lczerner@...hat.com>,
	linux-ext4@...r.kernel.org, Theodore Ts'o <tytso@....edu>
Subject: Re: Proposal draft for data checksumming for ext4

On Apr 28, 2014, at 10:21 AM, Dmitry Monakhov <dmonakhov@...nvz.org> wrote:
>> On Thu, Mar 20, 2014 at 05:40:06PM +0100, Lukáš Czerner wrote:
>>> There are also other problems we should be concerned with. Ext4 file system
>>> does have support for metadata checksumming so all the metadata does have
>>> its own checksum. While we can avoid unnecessarily checksuming inodes, group
>>> descriptors and basicall all statically positioned metadata, we still have
>>> dynamically allocated metadata blocks such as extent blocks. These block
>>> do not have to be checksummed but we would still have space reserved in the
>>> checksum table.
>> 
>> Don't forget directory blocks--they (should) have checksums too, so you can
>> skip those.
> 
> Just quick note: We can hide checksum for directory inside
> ext4_dir_entry_2 for a special dirs '.' or '..' simply by increasing
> ->rec_len which make this feature compatible with older FS

First note - the htree index information for each directory is already
stored after the ".." entry in block 0 of the directory.

Also note that there is also a feature we developed for Lustre named
"dirdata" (EXT4_FEATURE_INCOMPAT_DIRDATA is already reserved) that
allows storing more information with each directory entry[*].  We
use this to store a 128-bit identifier with each object so that it is
unique across the cluster, so it is available efficiently for readdir.

We've needed to modify the handling of the htree index data so that
it properly skips the extended directory entry (patch attached), and
this also cleans up this code a bit to conform to modern coding style.
Without this patch, the htree code assumes that the dx_info struct is
immediately following hard-coded "." and ".." entries and does not
actually check the de->rec_len for these entries to determine the right
amount of data to skip.

The patch is based on an older kernel and may not apply directly to
mainline, but is required for any similar changes in this area.

That said, I think that Darrick's metadata checksum patches already
store per-directory-block checksums at the end of the directory in
a dummy entry, so I'm not sure what else is needed here?

Cheers, Andreas

[*] for reference the dirdata patch is at http://git.hpdd.intel.com/?p=fs/lustre-release.git;a=blob;f=ldiskfs/kernel_patches/patches/sles11sp2/ext4-data-in-dirent.patch

Download attachment "ext4-kill-dx_root.patch" of type "application/octet-stream" (7043 bytes)

Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)