| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <167344725490.2425628.13771289553670112965.stgit@warthog.procyon.org.uk>
Date: Wed, 11 Jan 2023 14:27:35 +0000
From: David Howells <dhowells@...hat.com>
To: Al Viro <viro@...iv.linux.org.uk>
Cc: linux-fsdevel@...r.kernel.org, linux-mm@...ck.org,
Rohith Surabattula <rohiths.msft@...il.com>,
linux-block@...r.kernel.org, Matthew Wilcox <willy@...radead.org>,
Jeff Layton <jlayton@...nel.org>, linux-cachefs@...hat.com,
Jan Kara <jack@...e.cz>, Logan Gunthorpe <logang@...tatee.com>,
Christoph Hellwig <hch@....de>, Jens Axboe <axboe@...nel.dk>,
Steve French <sfrench@...ba.org>,
Shyam Prasad N <nspmangalore@...il.com>,
John Hubbard <jhubbard@...dia.com>, linux-cifs@...r.kernel.org,
dhowells@...hat.com, Christoph Hellwig <hch@...radead.org>,
Matthew Wilcox <willy@...radead.org>,
Jens Axboe <axboe@...nel.dk>, Jan Kara <jack@...e.cz>,
Jeff Layton <jlayton@...nel.org>,
Logan Gunthorpe <logang@...tatee.com>,
linux-fsdevel@...r.kernel.org, linux-block@...r.kernel.org,
linux-kernel@...r.kernel.org
Subject: [PATCH v5 0/9] iov_iter: Add extraction helpers
Hi Al,
Here are patches clean up some use of READ/WRITE and ITER_SOURCE/DEST,
patches to provide support for extracting pages from an iov_iter and a
patch to use the primary extraction function in the block layer bio code if
you could take a look?
[!] NOTE that I've switched the functions to be exported GPL-only at
Christoph's request[1]. They are, however, intended as a replacement
for iov_iter_get_pages*(), which is not marked _GPL - so that
functionality will probably become unavailable to non-GPL 3rd party
modules in future.
The first three patches deal with ITER_SOURCE/DEST:
(1) Switch ITER_SOURCE/DEST to an enum and add a couple of helper
functions to query if an iterator represents a source or a destination
buffer. Using an enum may allow extra consistency warnings from the
compiler.
(2) Use the ITER_SOURCE/DEST values in the iov_iter core functions rather
than READ/WRITE.
(3) Get rid of most of the callers of iov_iter_rw(), using the IOCB_WRITE
and IOMAP_WRITE instead where available. This leaves only two places
looking at the this value: a consistency check in cifs and a single
place in the block layer.
The next patch adds a replacement for iov_iter_get_pages*(), including
Logan's new version:
(4) Add a function to list-only, get or pin pages from an iterator as a
future replacement for iov_iter_get_pages*(). Pointers to the pages
are placed into an array (which will get allocated if not provided)
and, depending on the iterator type and direction, the pages will have
a ref or a pin get on them or be left untouched (on the assumption
that the caller manages their lifetime).
The determination is:
UBUF/IOVEC + DEST -> pin
UBUF/IOVEC + SOURCE -> get
PIPE + DEST -> list-only
BVEC/XARRAY -> list-only
Anything else -> EFAULT
The function also returns an indication of which of "list only, get or
pin" the extraction function did to aid in cleaning up (returning 0,
FOLL_GET or FOLL_PIN as appropriate).
Then there are a couple of patches that add stuff to netfslib that I want
to use there as well as in cifs:
(5) Add a netfslib function to use (4) to extract pages from an ITER_IOBUF
or ITER_UBUF iterator into an ITER_BVEC iterator. This will get or
pin the pages as appropriate.
(6) Add a netfslib function to extract pages from an iterator that's of
type ITER_UBUF/IOVEC/BVEC/KVEC/XARRAY and add them to a scatterlist.
The function in (4) is used for a UBUF and IOVEC iterators, so those
need cleaning up afterwards. BVEC and XARRAY iterators are ungot and
unpinned and may be rendered into elements that span multiple pages,
for example if large folios are present.
Finally, there are a set of three patches to make the block layer's BIO
code use iov_iter_extract_pages():
(7) Replace BIO_NO_PAGE_REF with BIO_PAGE_REFFED (opposite logic).
(8) Make the block layer's BIO code pin pages or leave pages unaltered
rather than getting a ref on the pages when the circumstances warrant,
and noting in the bio struct what cleanups should be performed so that
the bio cleanup code then does the right thing.
(9) Remove an unnecessary check against 0 in bio_flagged() (it returns
bool) thus allowing the gcc optimiser to combine multiple instances of
the bitwise-AND on the same flags value.
Changes:
========
ver #5)
- Replace BIO_NO_PAGE_REF with BIO_PAGE_REFFED and split into own patch.
- Transcribe FOLL_GET/PIN into BIO_PAGE_REFFED/PINNED flags.
- Add patch to allow bio_flagged() to be combined by gcc.
ver #4)
- Drop the patch to move the FOLL_* flags to linux/mm_types.h as they're
no longer referenced by linux/uio.h.
- Add ITER_SOURCE/DEST cleanup patches.
- Make iov_iter/netfslib iter extraction patches use ITER_SOURCE/DEST.
- Allow additional gup_flags to be passed into iov_iter_extract_pages().
- Add struct bio patch.
ver #3)
- Switch to using EXPORT_SYMBOL_GPL to prevent indirect 3rd-party access
to get/pin_user_pages_fast()[1].
ver #2)
- Rolled the extraction cleanup mode query function into the extraction
function, returning the indication through the argument list.
- Fixed patch 4 (extract to scatterlist) to actually use the new
extraction API.
I've pushed the patches here also:
https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/log/?h=iov-extract
David
Link: https://lore.kernel.org/r/Y3zFzdWnWlEJ8X8/@infradead.org/ [1]
Link: https://lore.kernel.org/r/166697254399.61150.1256557652599252121.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/166722777223.2555743.162508599131141451.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/166732024173.3186319.18204305072070871546.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/166869687556.3723671.10061142538708346995.stgit@warthog.procyon.org.uk/ # rfc
Link: https://lore.kernel.org/r/166920902005.1461876.2786264600108839814.stgit@warthog.procyon.org.uk/ # v2
Link: https://lore.kernel.org/r/166997419665.9475.15014699817597102032.stgit@warthog.procyon.org.uk/ # v3
Link: https://lore.kernel.org/r/167305160937.1521586.133299343565358971.stgit@warthog.procyon.org.uk/ # v4
---
David Howells (9):
iov_iter: Change the direction macros into an enum
iov_iter: Use the direction in the iterator functions
iov_iter: Use IOCB/IOMAP_WRITE if available rather than iterator direction
iov_iter: Add a function to extract a page list from an iterator
netfs: Add a function to extract a UBUF or IOVEC into a BVEC iterator
netfs: Add a function to extract an iterator into a scatterlist
bio: Rename BIO_NO_PAGE_REF to BIO_PAGE_REFFED and invert the meaning
iov_iter, block: Make bio structs pin pages rather than ref'ing if appropriate
bio: Fix bio_flagged() so that it can be combined
block/bio.c | 59 ++++--
block/fops.c | 8 +-
fs/9p/vfs_addr.c | 2 +-
fs/affs/file.c | 4 +-
fs/ceph/file.c | 2 +-
fs/dax.c | 6 +-
fs/direct-io.c | 22 +--
fs/exfat/inode.c | 6 +-
fs/ext2/inode.c | 2 +-
fs/f2fs/file.c | 10 +-
fs/fat/inode.c | 4 +-
fs/fuse/dax.c | 2 +-
fs/fuse/file.c | 8 +-
fs/hfs/inode.c | 2 +-
fs/hfsplus/inode.c | 2 +-
fs/iomap/direct-io.c | 6 +-
fs/jfs/inode.c | 2 +-
fs/netfs/Makefile | 1 +
fs/netfs/iterator.c | 367 ++++++++++++++++++++++++++++++++++
fs/nfs/direct.c | 2 +-
fs/nilfs2/inode.c | 2 +-
fs/ntfs3/inode.c | 2 +-
fs/ocfs2/aops.c | 2 +-
fs/orangefs/inode.c | 2 +-
fs/reiserfs/inode.c | 2 +-
fs/udf/inode.c | 2 +-
include/linux/bio.h | 5 +-
include/linux/blk_types.h | 3 +-
include/linux/netfs.h | 7 +
include/linux/uio.h | 59 ++++--
lib/iov_iter.c | 407 +++++++++++++++++++++++++++++++++++---
mm/vmalloc.c | 1 +
32 files changed, 903 insertions(+), 108 deletions(-)
create mode 100644 fs/netfs/iterator.c
Powered by blists - more mailing lists