linux-kernel - Re: [GIT PULL] bcachefs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230906161002.2ztelmyzgy3pbmcd@moria.home.lan>
Date:   Wed, 6 Sep 2023 12:10:02 -0400
From:   Kent Overstreet <kent.overstreet@...ux.dev>
To:     Matthew Wilcox <willy@...radead.org>
Cc:     Christoph Hellwig <hch@...radead.org>,
        torvalds@...ux-foundation.org, linux-kernel@...r.kernel.org,
        linux-fsdevel@...r.kernel.org, linux-bcachefs@...r.kernel.org,
        Christian Brauner <christian@...uner.io>
Subject: Re: [GIT PULL] bcachefs

On Wed, Sep 06, 2023 at 01:41:22AM +0100, Matthew Wilcox wrote:
> On Tue, Sep 05, 2023 at 08:00:07PM -0400, Kent Overstreet wrote:
> > On Tue, Sep 05, 2023 at 06:24:47AM -0700, Christoph Hellwig wrote:
> > > Hi Kent,
> > > 
> > > I thought you'd follow Christians proposal to actually work with
> > > people to try to use common APIs (i.e. to use iomap once it's been
> > > even more iter-like, for which I'm still waiting for suggestions),
> > > and make your new APIs used more widely if they are a good idea
> > > (which also requires explaining them better) and aim for 6.7 once
> > > that is done.
> > 
> > Christoph, I get that iomap is your pet project and you want to make it
> > better and see it more widely used.
> > 
> > But the reasons bcachefs doesn't use iomap have been discussed at
> > length, and I've posted and talked about the bcachefs equivalents of
> > that code. You were AWOL on those discussions; you consistently say
> > "bcachefs should use iomap" and then walk away, so those discussions
> > haven't moved forwards.
> > 
> > To recap, besides being more iterator like (passing data structures
> > around with iterators, vs. indirect function calls into the filesystem),
> > bcachefs also hangs a bit more state off the pagecache, due to being
> > multi device and also using the same data structure for tracking disk
> > reservations (because why make the buffered write paths look that up
> > separately?).
> 
> I /thought/ the proposal was to use iomap for bcachefs DIO and leave
> buffered writes for a different day.  I agree the iomap buffered write
> path is inappropriate for bcachefs today.  I'd like that to change,
> but there's a lot of functionality that it would need to support.

No, I'm not going to convert the bcachefs DIO path to iomap; not as it
exitss now.

Right now I've got a clean separation between the VFS level DIO code,
and the lower level bcachefs code that walks the extents btree and
issues IOs. I have to consider the iomap approach where the
loop-up-mappings-and-issue loop is in iomap code but calling into
filesystem code pretty gross.

I was talking about this /years/ ago when I did the work to make it
possible to efficiently split bios - iomap could have taken the approach
bcachefs did, the prereqs were in place when iomap was started, but it
didn't happen - iomap ended up being a more conservative approach, a
cleaned up version of buffer heads and fs/direct-IO.c.

That's fine, iomap is certainly an improvement over what it was
replacing, but it would not be an improvement for bcachefs.

I think it might be more fruitful to look at consolidating the buffered
IO code in bcachefs and iomap. The conceptual models are a bit closer,
bcachefs's buffered IO code is just a bit more fully-featured in that it
does the dirty block tracking in a unified way. That was a project that
turned out pretty nicely.