| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20240607143919.2622319-1-john.g.garry@oracle.com>
Date: Fri, 7 Jun 2024 14:38:57 +0000
From: John Garry <john.g.garry@...cle.com>
To: axboe@...nel.dk, tytso@....edu, dchinner@...hat.com,
viro@...iv.linux.org.uk, brauner@...nel.org, djwong@...nel.org,
jack@...e.com, chandan.babu@...cle.com, hch@....de
Cc: linux-block@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-btrfs@...r.kernel.org, linux-erofs@...ts.ozlabs.org,
linux-ext4@...r.kernel.org, linux-f2fs-devel@...ts.sourceforge.net,
linux-fsdevel@...r.kernel.org, gfs2@...ts.linux.dev,
linux-xfs@...r.kernel.org, catherine.hoang@...cle.com,
ritesh.list@...il.com, mcgrof@...nel.org,
mikulas@...ax.karlin.mff.cuni.cz, agruenba@...hat.com,
miklos@...redi.hu, martin.petersen@...cle.com,
John Garry <john.g.garry@...cle.com>
Subject: [PATCH v4 00/22] block atomic writes for xfs
This series expands atomic write support to filesystems, specifically
XFS. Extent alignment is based on new feature forcealign.
Flag FS_XFLAG_ATOMICWRITES is added as an enabling flag for atomic writes.
XFS can be formatted for atomic writes as follows:
mkfs.xfs -i forcealign=1 -d extsize=16384 -d atomic-writes=1 /dev/sda
atomic-writes=1 just enables atomic writes in the SB, but does not auto-
enable atomic writes for each file.
Support can be enabled through xfs_io command:
$xfs_io -c "lsattr -v" filename
[extsize, force-align]
$xfs_io -c "extsize" filename
[16384] filename
$xfs_io -c "chattr +W" filename
$xfs_io -c "lsattr -v" filename
[extsize, force-align, atomic-writes] filename
$xfs_io -c statx filename
...
stat.stx_atomic_write_unit_min = 4096
stat.stx_atomic_write_unit_max = 16384
stat.stx_atomic_write_segments_max = 1
...
A known issue - as reported in
https://urldefense.com/v3/__https://lore.kernel.org/linux-xfs/20240429174746.2132161-1-john.g.garry@oracle.com/T/*m7093bc85a8e0cbe13c111284768476d294aa077a__;Iw!!ACWV5N9M2RV99hQ!NbuQfXN8ZuUf_an3A6jHUXg3L1oCzefzyTYl0QWgJP1WbQCO8J_NPT9GHdGothSf36d0vxzJAjVUvcIB6IoU9nq3XExF$
-
is that forcealign is broken for !power-of-2 sizes. That needs fixing.
New in this series is a re-work of the iomap extent granularity zeroing
code. In the earlier series, iomap would consider a larger block zeroing
size when a member is set in struct iomap. Now each fs is responsible for
setting this size, which is i_blocksize(inode) when we just want regular
sub-fs block zeroing. All relevant FSes which use iomap are fixing up for
this.
Baseline is following series (which is based on Jens' block-6.10 branch):
https://lore.kernel.org/linux-nvme/96cb2069-a8e2-4723-802c-3ad4ba3e3d42@oracle.com/T/#mb980c084be402472601831c47fb2b66d0bfa8f0e
Basic xfsprogs support at:
https://github.com/johnpgarry/xfsprogs-dev/tree/forcealign_and_atomicwrites_for_v4_xfs_block_atomic_writes
Patches for this series can be found at:
https://github.com/johnpgarry/linux/commits/atomic-writes-v6.10-v7-fs-v4/
Changes since v3:
https://lore.kernel.org/linux-xfs/20240429174746.2132161-1-john.g.garry@oracle.com/T/#m9424b3cd1ccfde795d04474fdb4456520b6b4242
- Only enforce forcealign extsize is power-of-2 for atomic writes
- Re-org some validation code
- Fix xfs_bmap_process_allocated_extent() for forcealign
- Support iomap->io_block_size and make each fs support it
- Add !power-of-2 iomap support for io_block_size
- Make iomap dio iter handle atomic write failure properly by zeroing the
remaining io_block_size
Changes since v2:
https://lore.kernel.org/linux-xfs/20240304130428.13026-1-john.g.garry@oracle.com/
- Incorporate forcealign patches from
https://lore.kernel.org/linux-xfs/20240402233006.1210262-1-david@fromorbit.com/
- Put bdev awu min and max in buftarg
- Extra forcealign patches to deal with truncate and fallocate punch,
insert, collapse
- Add generic_atomic_write_valid_size()
- Change iomap.extent_shift -> .extent_size
Darrick J. Wong (2):
xfs: Introduce FORCEALIGN inode flag
xfs: Enable file data forcealign feature
Dave Chinner (6):
xfs: only allow minlen allocations when near ENOSPC
xfs: always tail align maxlen allocations
xfs: simplify extent allocation alignment
xfs: make EOF allocation simpler
xfs: introduce forced allocation alignment
xfs: align args->minlen for forced allocation alignment
John Garry (14):
fs: Add generic_atomic_write_valid_size()
iomap: Allow filesystems set IO block zeroing size
xfs: Use extent size granularity for iomap->io_block_size
xfs: Do not free EOF blocks for forcealign
xfs: Update xfs_inode_alloc_unitsize_fsb() for forcealign
xfs: Unmap blocks according to forcealign
xfs: Only free full extents for forcealign
xfs: Don't revert allocated offset for forcealign
fs: Add FS_XFLAG_ATOMICWRITES flag
iomap: Atomic write support
xfs: Support FS_XFLAG_ATOMICWRITES for forcealign
xfs: Support atomic write for statx
xfs: Validate atomic writes
xfs: Support setting FMODE_CAN_ATOMIC_WRITE
block/fops.c | 1 +
fs/btrfs/inode.c | 1 +
fs/erofs/data.c | 1 +
fs/erofs/zmap.c | 1 +
fs/ext2/inode.c | 1 +
fs/ext4/extents.c | 1 +
fs/ext4/inode.c | 1 +
fs/f2fs/data.c | 1 +
fs/fuse/dax.c | 1 +
fs/gfs2/bmap.c | 1 +
fs/hpfs/file.c | 1 +
fs/iomap/direct-io.c | 41 ++++-
fs/xfs/libxfs/xfs_alloc.c | 33 ++--
fs/xfs/libxfs/xfs_alloc.h | 3 +-
fs/xfs/libxfs/xfs_bmap.c | 308 ++++++++++++++++++----------------
fs/xfs/libxfs/xfs_format.h | 16 +-
fs/xfs/libxfs/xfs_ialloc.c | 12 +-
fs/xfs/libxfs/xfs_inode_buf.c | 105 ++++++++++++
fs/xfs/libxfs/xfs_inode_buf.h | 5 +
fs/xfs/libxfs/xfs_sb.c | 4 +
fs/xfs/xfs_bmap_util.c | 14 +-
fs/xfs/xfs_buf.c | 15 +-
fs/xfs/xfs_buf.h | 4 +-
fs/xfs/xfs_buf_mem.c | 2 +-
fs/xfs/xfs_file.c | 49 +++++-
fs/xfs/xfs_inode.c | 41 ++++-
fs/xfs/xfs_inode.h | 29 ++++
fs/xfs/xfs_ioctl.c | 83 ++++++++-
fs/xfs/xfs_iomap.c | 7 +
fs/xfs/xfs_iops.c | 28 ++++
fs/xfs/xfs_mount.h | 4 +
fs/xfs/xfs_reflink.h | 10 --
fs/xfs/xfs_super.c | 8 +
fs/xfs/xfs_trace.h | 8 +-
fs/zonefs/file.c | 2 +
include/linux/fs.h | 12 ++
include/linux/iomap.h | 2 +
include/uapi/linux/fs.h | 3 +
38 files changed, 656 insertions(+), 203 deletions(-)
--
2.31.1
Powered by blists - more mailing lists