linux-kernel - Re: [GIT PULL] bcachefs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20230706155602.mnhsylo3pnief2of@moria.home.lan>
Date:   Thu, 6 Jul 2023 11:56:02 -0400
From:   Kent Overstreet <kent.overstreet@...ux.dev>
To:     torvalds@...ux-foundation.org
Cc:     linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-bcachefs@...r.kernel.org, djwong@...nel.org,
        dchinner@...hat.com, sandeen@...hat.com, willy@...radead.org,
        josef@...icpanda.com, tytso@....edu, bfoster@...hat.com,
        jack@...e.cz, andreas.gruenbacher@...il.com, brauner@...nel.org,
        peterz@...radead.org, akpm@...ux-foundation.org,
        dhowells@...hat.com
Subject: Re: [GIT PULL] bcachefs

On Mon, Jun 26, 2023 at 05:47:01PM -0400, Kent Overstreet wrote:
> Hi Linus,
> 
> Here it is, the bcachefs pull request. For brevity the list of patches
> below is only the initial part of the series, the non-bcachefs prep
> patches and the first bcachefs patch, but the diffstat is for the entire
> series.
> 
> Six locks has all the changes you suggested, text size went down
> significantly. If you'd still like this to see more review from the
> locking people, I'm not against them living in fs/bcachefs/ as an
> interim; perhaps Dave could move them back to kernel/locking when he
> starts using them or when locking people have had time to look at them -
> I'm just hoping for this to not block the merge.
> 
> Recently some people have expressed concerns about "not wanting a repeat
> of ntfs3" - from what I understand the issue there was just severe
> buggyness, so perhaps showing the bcachefs automated test results will
> help with that:
> 
>   https://evilpiepirate.org/~testdashboard/ci
> 
> The main bcachefs branch runs fstests and my own test suite in several
> varations, including lockdep+kasan, preempt, and gcov (we're at 82% line
> coverage); I'm not currently seeing any lockdep or kasan splats (or
> panics/oopses, for that matter).
> 
> (Worth noting the bug causing the most test failures by a wide margin is
> actually an io_uring bug that causes random umount failures in shutdown
> tests. Would be great to get that looked at, it doesn't just affect
> bcachefs).
> 
> Regarding feature status - most features are considered stable and ready
> for use, snapshots and erasure coding are both nearly there. But a
> filesystem on this scale is a massive project, adequately conveying the
> status of every feature would take at least a page or two.
> 
> We may want to mark it as EXPERIMENTAL for a few releases, I haven't
> done that as yet. (I wouldn't consider single device without snapshots
> to be experimental, but - given that the number of users and bug reports
> is about to shoot up, perhaps I should...).

Restarting the discussion after the holiday weekend, hoping to get
something more substantive going:

Hoping to get:
 - Thoughts from people who have been following bcachefs development,
   and people who have looked at the code
 - Continuation of the LSF discussion - maybe some people could repeat
   here what they said there (re: code review, iomap, etc.)
 - Any concerns about how this might impact the rest of the kernel, or
   discussion about what impact merging a new filesystem is likely to
   have on other people's work

AFAIK the only big ask that hasn't happened yet is better documentation:
David Howells wanted (better) a man page, which is definitely something
that needs to happen but it'll be some months before I'm back to working
on documentation - I'm happy to share my current list of priorities if
that would be helpful.

In the meantime, the Latex principles of operation is reasonably up to
date (and I intend to greatly expand the sections on on disk data
structures, I think that'll be great reference documentation for
developers getting up to speed on the code)

https://bcachefs.org/bcachefs-principles-of-operation.pdf

I feel that bcachefs is in a pretty mature state at this point, but it's
also _huge_, which is a bit different than e.g. the btrfs merger; it's
hard to know where to start to get a meaninful discussion/review process
going.

Patch bombing the mailing list with 90k loc is clearly not going to be
productive, which is why I've been trying to talk more about development
process and status - but all suggestions and feedback are welcome.

Cheers,
Kent