[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20230905214235.320571-2-peterx@redhat.com>
Date: Tue, 5 Sep 2023 17:42:29 -0400
From: Peter Xu <peterx@...hat.com>
To: linux-kernel@...r.kernel.org, linux-mm@...ck.org
Cc: Anish Moorthy <amoorthy@...gle.com>,
Axel Rasmussen <axelrasmussen@...gle.com>,
Alexander Viro <viro@...iv.linux.org.uk>,
Mike Kravetz <mike.kravetz@...cle.com>,
Peter Zijlstra <peterz@...radead.org>,
Andrew Morton <akpm@...ux-foundation.org>,
Mike Rapoport <rppt@...nel.org>,
Christian Brauner <brauner@...nel.org>, peterx@...hat.com,
linux-fsdevel@...r.kernel.org,
Andrea Arcangeli <aarcange@...hat.com>,
Ingo Molnar <mingo@...hat.com>,
James Houghton <jthoughton@...gle.com>,
Nadav Amit <nadav.amit@...il.com>
Subject: [PATCH 1/7] mm/userfaultfd: Make uffd read() wait event exclusive
From: Andrea Arcangeli <aarcange@...hat.com>
When a new message is generated for an userfaultfd, instead of waking up
all the readers, we can wake up only one exclusive reader to process the
event. Waking up >1 readers for 1 message will be a waste of resource,
where the rest readers will see nothing again and re-queue.
This should make userfaultfd read() O(1) on wakeups.
Note that queuing on head is intended (rather than tail) to make sure the
readers are waked up in LIFO fashion; fairness doesn't matter much here,
but caching does.
Signed-off-by: Andrea Arcangeli <aarcange@...hat.com>
[peterx: modified subjects / commit message]
Signed-off-by: Peter Xu <peterx@...hat.com>
---
fs/userfaultfd.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c
index 56eaae9dac1a..f7fda7d0c994 100644
--- a/fs/userfaultfd.c
+++ b/fs/userfaultfd.c
@@ -1061,7 +1061,11 @@ static ssize_t userfaultfd_ctx_read(struct userfaultfd_ctx *ctx, int no_wait,
/* always take the fd_wqh lock before the fault_pending_wqh lock */
spin_lock_irq(&ctx->fd_wqh.lock);
- __add_wait_queue(&ctx->fd_wqh, &wait);
+ /*
+ * Only wake up one exclusive reader each time there's an event.
+ * Paired with wake_up_poll() when e.g. a new page fault msg generated.
+ */
+ __add_wait_queue_exclusive(&ctx->fd_wqh, &wait);
for (;;) {
set_current_state(TASK_INTERRUPTIBLE);
spin_lock(&ctx->fault_pending_wqh.lock);
--
2.41.0
Powered by blists - more mailing lists