linux-ext4 - [PATCH v5 0/7] ext4: Add multi-fsblock atomic write support with bigalloc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <cover.1747337952.git.ritesh.list@gmail.com>
Date: Fri, 16 May 2025 01:20:48 +0530
From: "Ritesh Harjani (IBM)" <ritesh.list@...il.com>
To: linux-ext4@...r.kernel.org
Cc: Theodore Ts'o <tytso@....edu>,
	Jan Kara <jack@...e.cz>,
	John Garry <john.g.garry@...cle.com>,
	djwong@...nel.org,
	Ojaswin Mujoo <ojaswin@...ux.ibm.com>,
	linux-fsdevel@...r.kernel.org,
	"Ritesh Harjani (IBM)" <ritesh.list@...il.com>
Subject: [PATCH v5 0/7] ext4: Add multi-fsblock atomic write support with bigalloc

This adds multi-fsblock atomic write support to ext4 using bigalloc. The major
chunk of the design changes are kept in Patch-4 & 5.

v4 -> v5:
=========
1. Addressed review comments and added Acked-by from Darrick.
2. Changed a minor WARN_ON(1) at one place to WARN_ON_ONCE(1) in patch-5 in
   ext4_iomap_begin(). Ideally we may never hit it.
3. Added force commit related info in the Documentation section where
   mixed mapping details are mentioned.
[v4]: https://lore.kernel.org/linux-ext4/cover.1747289779.git.ritesh.list@gmail.com/

v3 -> v4:
=========
1. Added force txn commit in ext4_iomap_alloc(), in case we ended up allocating
   and converting unwritten to written conversion during allocation time. This
   happens when the underlying region has mixed mapping [1].
2. Addressed review comments from Darrick and Ojaswin.
3. Ran few data integrity tests on actual nvme device supporting atomic writes.

[v2]: https://lore.kernel.org/linux-ext4/cover.1746734745.git.ritesh.list@gmail.com/
[1]: https://lore.kernel.org/linux-ext4/87msbfyqcm.fsf@gmail.com/T/#m9f6607cc0e65e9fd29e675c5c15b3a03ff02110e

v2 -> v3:
=========
1. Improved error handling at several places.
2. Further fixed some worst case journal credits estimation.
3. Added better checks in the slow path allocation loop for atomic writes.

v3 testing so far:
===============
- This has survived "quick" & "auto" group testing with bigalloc on x86 and Power.
- We have also tested atomic write related tests using fio and some data integrity
  tests with sudden power off during writes on scsi_debug module.
  (Will clean up these tests and try to post them out soon!)

Appreciate any review comments / feedback!

v1 -> v2:
==========
1. Handled review comments from Ojaswin to optimize the ext4_map_block() calls
   in ext4_iomap_alloc().
2. Fixed the journal credits calculation for both:
	- during block allocation in ext4_iomap_alloc()
	- during dio completion in ->end_io callback.
   Earlier we were starting multiple txns in ->end_io callback for unwritten to
   written conversion. But since in case of atomic writes, we want a single jbd2
   txn, hence made the necessary changes there.
[v2]: https://lore.kernel.org/linux-ext4/cover.1745987268.git.ritesh.list@gmail.com/

Ritesh Harjani (IBM) (7):
  ext4: Document an edge case for overwrites
  ext4: Check if inode uses extents in ext4_inode_can_atomic_write()
  ext4: Make ext4_meta_trans_blocks() non-static for later use
  ext4: Add support for EXT4_GET_BLOCKS_QUERY_LEAF_BLOCKS
  ext4: Add multi-fsblock atomic write support with bigalloc
  ext4: Enable support for ext4 multi-fsblock atomic write using bigalloc
  ext4: Add atomic block write documentation

 .../filesystems/ext4/atomic_writes.rst        | 225 +++++++++++++
 Documentation/filesystems/ext4/overview.rst   |   1 +
 fs/ext4/ext4.h                                |  26 +-
 fs/ext4/extents.c                             |  99 ++++++
 fs/ext4/file.c                                |   7 +-
 fs/ext4/inode.c                               | 315 ++++++++++++++++--
 fs/ext4/super.c                               |   7 +-
 7 files changed, 655 insertions(+), 25 deletions(-)
 create mode 100644 Documentation/filesystems/ext4/atomic_writes.rst

--
2.49.0