linux-ext4 - Re: [PATCH v5 0/10] fs: Introduce new flag(FALLOC_FL_COLLAPSE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20140225175216.0f0c10f9.akpm@linux-foundation.org>
Date:	Tue, 25 Feb 2014 17:52:16 -0800
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Dave Chinner <david@...morbit.com>
Cc:	Hugh Dickins <hughd@...gle.com>,
	Namjae Jeon <linkinjeon@...il.com>,
	Matthew Wilcox <matthew@....cx>,
	"Theodore Ts'o" <tytso@....edu>,
	Stephen Rothwell <sfr@...b.auug.org.au>,
	viro@...iv.linux.org.uk, bpm@....com, adilger.kernel@...ger.ca,
	jack@...e.cz, mtk.manpages@...il.com, lczerner@...hat.com,
	linux-fsdevel@...r.kernel.org, xfs@....sgi.com,
	linux-ext4@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, Namjae Jeon <namjae.jeon@...sung.com>
Subject: Re: [PATCH v5 0/10] fs: Introduce new
 flag(FALLOC_FL_COLLAPSE_RANGE) for fallocate

On Wed, 26 Feb 2014 12:34:26 +1100 Dave Chinner <david@...morbit.com> wrote:

> On Tue, Feb 25, 2014 at 03:41:28PM -0800, Andrew Morton wrote:
> > On Tue, 25 Feb 2014 15:23:35 -0800 (PST) Hugh Dickins <hughd@...gle.com> wrote:
> > > On Tue, 25 Feb 2014, Dave Chinner wrote:
> > > > On Tue, Feb 25, 2014 at 02:16:01PM +1100, Stephen Rothwell wrote:
> > > > > On Mon, 24 Feb 2014 11:57:10 +1100 Dave Chinner <david@...morbit.com> wrote:
> > > FALLOC_FL_COLLAPSE_RANGE: I'm a little sad at the name COLLAPSE,
> > > but probably seven months too late to object.  It surprises me that
> > > you're doing all this work to deflate a part of the file, without
> > > the obvious complementary work to inflate it - presumably all those
> > > advertisers whose ads you're cutting out, will come back to us soon
> > > to ask for inflation, so that they have somewhere to reinsert them ;)
> > 
> > Yes, I was wondering that.  Why not simply "move these blocks from here
> > to there".
> 
> And open a completely unnecessary can of worms to do with
> behavioural and implementation corner cases?

But it's general.

> Do you allow it to destroy data by default? Or only allow moves into
> holes?

Overwrite.

> What do you do with range the data is moved out of? Does it just
> become a hole? What happens if the range overlaps EOF - does that
> change the file size?

Truncate.

> What if you want to move the range beyond EOF?

Extend.

> What if the source and destination ranges overlap?

Don't screw it up.

> What happens when you move the block at EOF into the middle of a
> file - do you end up with zeros padding the block and the file size
> having to be adjusted accordingly? Or do we have to *copy* all the
> data in high blocks down to fill the hole in the block?

I don't understand that.  Move the block(s) and truncate to the new
length.

> What behaviour should we expect if the filesystem can't implement
> the entire move atomically and we crash in the middle of the move?

What does collapse_range do now?

If it's a journaled filesystem, it shouldn't screw up.  If it isn't, fsck.

> I can keep going, but I'll stop here - you get the idea.

None of this seems like rocket science.

> In comparison, collapse range as a file data manipulation has very
> specific requirements and from that we can define a simple, specific
> API that allows filesystems to accelerate that operation by extent
> manipulation rather than read/memcpy/write that the applications are
> currently doing for this operation....  IOWs, collapse range is a
> simple operation, "move arbitrary blocks from here to there" is a
> nightmare both from the specification and the implementation points
> of view.

collapse_range seems weird, arbitrary and half-assed.  "Why didn't they
go all the way and do it properly".

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html