linux-ext4 - Re: [dm-devel] Some thoughts about providing data block checksumming for ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20141105023337.GA324@thunk.org>
Date:	Tue, 4 Nov 2014 21:33:37 -0500
From:	Theodore Ts'o <tytso@....edu>
To:	Mikulas Patocka <mpatocka@...hat.com>
Cc:	linux-ext4@...r.kernel.org, dm-devel@...hat.com
Subject: Re: [dm-devel] Some thoughts about providing data block checksumming
 for ext4

On Tue, Nov 04, 2014 at 04:39:55PM -0500, Mikulas Patocka wrote:
> 
> 
> On Mon, 3 Nov 2014, Theodore Ts'o wrote:
> 
> > But there is a way we can do even better!  If we can manage to
> > compress the block even by a tiny amount, so that 4k block can be
> > stored in 4092 bytes (which means we need to be able to compress the
> > block by 0.1%), we can store the checksum inline with the data, which
> > can then be atomically updated assuming a modern drive with a 4k
> > sector size (even a 512e disk will work fine, assuming the partition
> > is properly 4k aligned).  If the block is not sufficiently
> 
> There is still large number of drives with 512-byte sectors in use. So 
> we'd rather use 512-byte block?

There are a lot of systems (including Oracle IIRC) that use 4k blocks
and checksums, and accept the fact that very rarely it's possible that
even though writes are sent in chunks of 4k, it's possible (although
in general fairly rare) to have "torn writes" after a power failure. 

I'd much rather design for the future and not try to tie ourselves in
knots about the possibility of some torn writes on 512 byte sector
disks.  Many other file systems and databases have made similar
assumptions (and in fact have for years; I remember stories about
Oracle and another enterprise database having to deal with torn
writes eight years ago).

							- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html