linux-kernel - [PATCH 2/7] iomap: Move prefaulting out of hot write path

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20250129181753.3927F212@davehans-spike.ostc.intel.com>
Date: Wed, 29 Jan 2025 10:17:53 -0800
From: Dave Hansen <dave.hansen@...ux.intel.com>
To: linux-kernel@...r.kernel.org
Cc: Linus Torvalds <torvalds@...ux-foundation.org>,Ted Ts'o <tytso@....edu>,Christian Brauner <brauner@...nel.org>,Darrick J. Wong <djwong@...nel.org>,Matthew Wilcox (Oracle) <willy@...radead.org>,Al Viro <viro@...iv.linux.org.uk>,linux-fsdevel@...r.kernel.org,Dave Hansen <dave.hansen@...ux.intel.com>,linux-xfs@...r.kernel.org
Subject: [PATCH 2/7] iomap: Move prefaulting out of hot write path

From: Dave Hansen <dave.hansen@...ux.intel.com>

Prefaulting the write source buffer incurs an extra userspace access
in the common fast path. Make iomap_write_iter() consistent with
generic_perform_write(): only touch userspace an extra time when
copy_folio_from_iter_atomic() has failed to make progress.

Signed-off-by: Dave Hansen <dave.hansen@...ux.intel.com>
Cc: Christian Brauner <brauner@...nel.org>
Cc: "Darrick J. Wong" <djwong@...nel.org>
Cc: linux-xfs@...r.kernel.org
---

 b/fs/iomap/buffered-io.c |   24 +++++++++---------------
 1 file changed, 9 insertions(+), 15 deletions(-)

diff -puN fs/iomap/buffered-io.c~iomap-postfault fs/iomap/buffered-io.c
--- a/fs/iomap/buffered-io.c~iomap-postfault	2025-01-29 09:03:32.179361299 -0800
+++ b/fs/iomap/buffered-io.c	2025-01-29 09:03:32.187361965 -0800
@@ -937,21 +937,6 @@ retry:
 		if (bytes > length)
 			bytes = length;

-		/*
-		 * Bring in the user page that we'll copy from _first_.
-		 * Otherwise there's a nasty deadlock on copying from the
-		 * same page as we're writing to, without it being marked
-		 * up-to-date.
-		 *
-		 * For async buffered writes the assumption is that the user
-		 * page has already been faulted in. This can be optimized by
-		 * faulting the user page.
-		 */
-		if (unlikely(fault_in_iov_iter_readable(i, bytes) == bytes)) {
-			status = -EFAULT;
-			break;
-		}
-
 		status = iomap_write_begin(iter, pos, bytes, &folio);
 		if (unlikely(status)) {
 			iomap_write_failed(iter->inode, pos, bytes);
@@ -1005,6 +990,15 @@ retry:
 				bytes = copied;
 				goto retry;
 			}
+			/*
+			 * 'folio' is now unlocked and faults on it can be
+			 * handled. Ensure forward progress by trying to
+			 * fault it in now.
+			 */
+			if (fault_in_iov_iter_readable(i, bytes) == bytes) {
+				status = -EFAULT;
+				break;
+			}
 		} else {
 			pos += written;
 			total_written += written;
_