| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20230929102726.2985188-1-john.g.garry@oracle.com>
Date: Fri, 29 Sep 2023 10:27:05 +0000
From: John Garry <john.g.garry@...cle.com>
To: axboe@...nel.dk, kbusch@...nel.org, hch@....de, sagi@...mberg.me,
jejb@...ux.ibm.com, martin.petersen@...cle.com, djwong@...nel.org,
viro@...iv.linux.org.uk, brauner@...nel.org,
chandan.babu@...cle.com, dchinner@...hat.com
Cc: linux-block@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-nvme@...ts.infradead.org, linux-xfs@...r.kernel.org,
linux-fsdevel@...r.kernel.org, tytso@....edu, jbongio@...gle.com,
linux-api@...r.kernel.org, John Garry <john.g.garry@...cle.com>
Subject: [PATCH 00/21] block atomic writes
This series introduces a proposal to implementing atomic writes in the
kernel for torn-write protection.
This series takes the approach of adding a new "atomic" flag to each of
pwritev2() and iocb->ki_flags - RWF_ATOMIC and IOCB_ATOMIC, respectively.
When set, these indicate that we want the write issued "atomically".
Only direct IO is supported and for block devices and XFS.
The atomic writes feature requires dedicated HW support, like
SCSI WRITE_ATOMIC_16 command.
man pages update has been posted at:
https://lore.kernel.org/linux-api/20230929093717.2972367-1-john.g.garry@oracle.com/T/#t
The goal here is to provide an interface that allow applications use
application-specific block sizes larger than logical block size
reported by the storage device or larger than filesystem block size as
reported by stat().
With this new interface, application blocks will never be torn or
fractured when written. For a power fail, for each individual application
block, all or none of the data to be written. A racing atomic write and
read will mean that the read sees all the old data or all the new data,
but never a mix of old and new.
Two new fields are added to struct statx - atomic_write_unit_min and
atomic_write_unit_max. For each atomic individual write, the total length
of a write must be a between atomic_write_unit_min and
atomic_write_unit_max, inclusive, and a power-of-2. The write must also be
at a natural offset in the file wrt the write length.
For XFS, we must ensure extent alignment with the userspace block size.
XFS supports an extent size hint. However, it must be ensured that the
hint is honoured. For this, a new flag is added - forcealign - to
instruct the XFS block allocator to always honour the extent size hint.
The user would typically set the extent size hint at the userspace
block size to support atomic writes.
The atomic_write_unit_{min, max} values from statx on an XFS file will
consider both the backing bdev atomic_write_unit_{min, max} values and
the extent alignment for the file.
SCSI sd.c and scsi_debug and NVMe kernel support is added.
xfsprogs update for forcealign is at:
https://lore.kernel.org/linux-xfs/20230929095342.2976587-1-john.g.garry@oracle.com/T/#t
This series is based on v6.6-rc3.
Major changes since RFC (https://lore.kernel.org/linux-scsi/20230503183821.1473305-1-john.g.garry@oracle.com/):
- Add XFS forcealign feature
- Only allow writing a single userspace block
Alan Adamson (1):
nvme: Support atomic writes
Darrick J. Wong (3):
fs: xfs: Introduce FORCEALIGN inode flag
fs: xfs: Make file data allocations observe the 'forcealign' flag
fs: xfs: Enable file data forcealign feature
Himanshu Madhani (2):
block: Add atomic write operations to request_queue limits
block: Add REQ_ATOMIC flag
John Garry (13):
block: Limit atomic writes according to bio and queue limits
block: Pass blk_queue_get_max_sectors() a request pointer
block: Limit atomic write IO size according to
atomic_write_max_sectors
block: Error an attempt to split an atomic write bio
block: Add checks to merging of atomic writes
block: Add fops atomic write support
fs: xfs: Don't use low-space allocator for alignment > 1
fs: xfs: Support atomic write for statx
fs: iomap: Atomic write support
fs: xfs: iomap atomic write support
scsi: sd: Support reading atomic properties from block limits VPD
scsi: sd: Add WRITE_ATOMIC_16 support
scsi: scsi_debug: Atomic write support
Prasad Singamsetty (2):
fs/bdev: Add atomic write support info to statx
fs: Add RWF_ATOMIC and IOCB_ATOMIC flags for atomic write support
Documentation/ABI/stable/sysfs-block | 42 ++
block/bdev.c | 33 +-
block/blk-merge.c | 92 ++++-
block/blk-mq.c | 2 +-
block/blk-settings.c | 76 ++++
block/blk-sysfs.c | 33 ++
block/blk.h | 9 +-
block/fops.c | 42 +-
drivers/nvme/host/core.c | 29 ++
drivers/scsi/scsi_debug.c | 587 +++++++++++++++++++++------
drivers/scsi/scsi_trace.c | 22 +
drivers/scsi/sd.c | 57 ++-
drivers/scsi/sd.h | 7 +
fs/iomap/direct-io.c | 26 +-
fs/iomap/trace.h | 3 +-
fs/stat.c | 15 +-
fs/xfs/libxfs/xfs_bmap.c | 26 +-
fs/xfs/libxfs/xfs_format.h | 9 +-
fs/xfs/libxfs/xfs_inode_buf.c | 40 ++
fs/xfs/libxfs/xfs_inode_buf.h | 3 +
fs/xfs/libxfs/xfs_sb.c | 3 +
fs/xfs/xfs_inode.c | 12 +
fs/xfs/xfs_inode.h | 5 +
fs/xfs/xfs_ioctl.c | 18 +
fs/xfs/xfs_iomap.c | 40 +-
fs/xfs/xfs_iops.c | 51 +++
fs/xfs/xfs_iops.h | 4 +
fs/xfs/xfs_mount.h | 2 +
fs/xfs/xfs_super.c | 4 +
include/linux/blk_types.h | 2 +
include/linux/blkdev.h | 37 +-
include/linux/fs.h | 1 +
include/linux/iomap.h | 1 +
include/linux/stat.h | 2 +
include/scsi/scsi_proto.h | 1 +
include/trace/events/scsi.h | 1 +
include/uapi/linux/fs.h | 7 +-
include/uapi/linux/stat.h | 7 +-
38 files changed, 1179 insertions(+), 172 deletions(-)
--
2.31.1
Powered by blists - more mailing lists