linux-kernel - Re: Getting a new fs in the kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <YBD72wlZC323yhqZ@mit.edu>
Date:   Wed, 27 Jan 2021 00:36:27 -0500
From:   "Theodore Ts'o" <tytso@....edu>
To:     Chaitanya Kulkarni <Chaitanya.Kulkarni@....com>
Cc:     Amy Parker <enbyamy@...il.com>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: Getting a new fs in the kernel

On Tue, Jan 26, 2021 at 07:06:55PM +0000, Chaitanya Kulkarni wrote:
> From what I've seen you can post the long patch-series as an RFC and get the
> 
> discussion started.
> 
> The priority should be ease of review and not the total patch-count.

File systems are also complicated enough that it's useful to make the
patches available via a git repo, and it's highly recommended that you
are rebasing it against the latest kernel on a regular basis.

I also strongly recommend that once you get something that mostly
works, that you start doing regression testing of the file system.
Most of the major file systems in Linux use xfstests for their
testing.  One of the things that I've done is to package up xfstests
as a test appliance, suitable for running under KVM or using Google
Compute Engine, as a VM, to make it super easy for people to run
regression tests.  (One of my original goals for packaging it up was
to make it easy for graduate students who were creating research file
systems to try running regression tests so they could find potential
problems --- and understand how hard it is to make a robust,
production-ready file system, by giving them a realtively well
documented, turn-key system for running file system regression tests.)

For more information, see:

    https://thunk.org/gce-xfstests
    https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md
    https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-xfstests.md
    https://github.com/tytso/xfstests-bld/blob/master/Documentation/gce-xfstests.md

The final thing I'll point out is that file system development is a
team sport.  Industry estimates are that it takes between 50 and 200
person-years to create a production-ready, general purpose enterprise
file system.  For example, ZFS took seven years to develop, starting
with a core team of 4, and growing to over 14 developers by the time
it was announced.  And that didn't include all of the QA, release
engineering, testers, performance engineers, to get it integrated into
the Solaris product.  Even after it was announced, it was a good four
years before customers trusted it for production workloads.

If you look at the major file systems in Linux: ext4, xfs, btrfs,
f2fs, etc., you'll find that none of them are solo endeavors, and all
of them have multiple companies who are employing the developers who
work on them.  Figuring out how to convince companies that there are
good business reasons for them to support the developers of your file
system is important, since in order to keep things going for the long
haul, it really needs to be more than a single person's hobby.

Good luck!

					- Ted