linux-kernel - Cut down implementation of fscache new API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <160655.1611012999@warthog.procyon.org.uk>
Date:   Mon, 18 Jan 2021 23:36:39 +0000
From:   David Howells <dhowells@...hat.com>
To:     linux-fsdevel@...r.kernel.org, linux-cachefs@...hat.com
Cc:     dhowells@...hat.com, jlayton@...hat.com, dwysocha@...hat.com,
        Matthew Wilcox <willy@...radead.org>,
        Dominique Martinet <asmadeus@...ewreck.org>,
        Steve French <sfrench@...ba.org>,
        Trond Myklebust <trond.myklebust@...merspace.com>,
        Anna Schumaker <anna.schumaker@...app.com>,
        Christoph Hellwig <hch@....de>, dchinner@...hat.com,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        v9fs-developer@...ts.sourceforge.net,
        linux-afs@...ts.infradead.org, ceph-devel@...r.kernel.org,
        linux-cifs@...r.kernel.org, linux-nfs@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Cut down implementation of fscache new API

Take a look at:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/

I've extracted the netfs helper library from my patch set and built an
alternative cut-down I/O API for the existing fscache code as a bridge to
moving to a new fscache implementation.  With this, a netfs now has two
choices: use the existing API as is or use the netfs lib and the alternative
API.  You can't mix the two APIs - a netfs has to use one or the other.

It works with AFS, at least for reading data through a cache, and without a
cache, xfstests is quite happy.  I was able to take a bunch of the AFS patches
from my fscache-iter branch (the full rewrite) and apply them with minimal
changes.  Since it goes through the new I/O API in both cases, those changes
should be the same.  The main differences are in the cookie wrangling API.

The alternative API is different from the current in the following ways:

 (1) It uses kiocbs to do async DIO rather than using readpage() with page
     wake queue snooping and vfs_write().

 (2) It uses SEEK_HOLE/SEEK_DATA rather than bmap() to determine the location
     of data in the file.  This is still broken because we can't rely on this
     information in the backing filesystem.

 (3) It completely changes how PG_fscache is used.  As for the new API, it's
     used to indicate an in progress write to the cache from a page rather
     than a page the cache knows about.

 (4) It doesn't keep track of the netfs's pages beyond the termination of an
     I/O operation.  The old API added pages that have outstanding writes to
     the cache to a radix three for a background writer; now an async kiocb is
     dispatched.

 (5) The netfs needs to call fscache_begin_read_operation() from its
     ->begin_cache_operation() handler as passed to the netfs helper lib.
     This tells the netfs helpers how to access the cache.

 (6) It relies on the netfs helper lib to reissue a failed cache read to the
     server.

 (7) Handles THPs.

 (8) Implements completely ->readahead() and ->readpage() and implements a
     chunk of ->write_begin().

Things it doesn't address:

 (1) Mapping the content independently of the backing filesystem's metadata.

 (2) Getting rid of the backpointers into the netfs.

 (3) Simplifying the management of cookies and objects and their processing.

 (4) Holding an open file to the cache for any great length of time.  It gets
     a new file struct for each read op it does on the cache and drops it
     again afterwards.

 (5) Pinning the cache context/state required to handle a deferred write to
     the cache from ->write_begin() as performed by, say, ->writepages().

David