linux-ext4 - Re: [PATCH 1/4] vfs: vfs-level fiemap interface

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20080919173802.GB17666@shareable.org>
Date:	Fri, 19 Sep 2008 18:38:02 +0100
From:	Jamie Lokier <jamie@...reable.org>
To:	Chris Mason <chris.mason@...cle.com>
Cc:	Jörn Engel <joern@...fs.org>,
	Theodore Tso <tytso@....edu>, Andreas Dilger <adilger@....com>,
	Christoph Hellwig <hch@...radead.org>,
	linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org,
	akpm@...uxfoundation.org, Mark Fasheh <mfasheh@...e.com>,
	mtk.manpages@...il.com
Subject: Re: [PATCH 1/4] vfs: vfs-level fiemap interface

Chris Mason wrote:
> On Wed, Sep 17, 2008 at 04:02:12PM +0100, Jamie Lokier wrote:
> > Jörn Engel wrote:
> > > Apart from the typo above, here is a more discouraging version:
> > > 
> > >   In general, accessing the block device directly is strongly discouraged.
> > >   Exceptions exist mainly in the form of boot loaders like lilo and grub,
> > >   at a time when the filesystem is not (cannot be) mounted.
> > > 
> > >   If the flag DATA_ENCODED is set, however, even this exception is no
> > >   longer valid.  The content is encoded in some form.  Details are
> > >   unknown, it could be compressed, encrypted or something else.
> > 
> > I'm not clear about something from the above description.
> > 
> > If I were writing a journalling / tree-like filesystem, and I did
> > store data in blocks without encoding, but fsync() only waits for them
> > to be committed to journal, not their final destination, and also they
> > might be moved around - should I set DATA_ENCODED or not?  (And should
> > I return the temporary location in the long-running journal since
> > that's the only place the data is committed at the time of the call?)
> > 
> > Assume that even reading after unmounting is not 100% safe, because
> > the data blocks could be relocated after calling FIEMAP (when the
> > filesystem must be mounted), and before the unmount.
> 
> For the journal case at least, grub can walk through the log of the FS
> looking for up to date copies of things.  It does this already for
> reiserfs because the btree can't be trusted at all without a log replay.

Ok, that's good - grub doesn't need FIEMAP, it reads the filesystem properly.

So if I were writing a filesystem, what am I expected to return in
FIEMAP for these cases?  I'm thinking I should set DATA_ENCODED, even
though the examples in Jörn's description don't cover this.

I'm thinking there are three main uses for FIEMAP:

  1. LILO and similar.  LILO itself is fine with FIBMAP though.

  2. Fragmentation measurement and possibly defragmentation tools.

  3. Something wants to have an idea of which areas of disk will be
     accessed, so it can optimise I/O at a higher level - i.e. a database.
     This isn't foolproof, especially for writes on recent filesystems
     which don't overwrite in place.

1 means DATA_ENCODED should be set whenever there's any likelihood
that the result isn't reliable, so that would include when data is
stored in a journal or other temporary place and not a permanent place
on disk.

2 and 3 don't care about DATA_ENCODED at all.

-- Jamie
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html