[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <YpVEgWHzzH3ZtVzA@shell.armlinux.org.uk>
Date: Mon, 30 May 2022 23:26:09 +0100
From: "Russell King (Oracle)" <linux@...linux.org.uk>
To: Peter Xu <peterx@...hat.com>
Cc: linux-kernel@...r.kernel.org, linux-mm@...ck.org,
Ingo Molnar <mingo@...hat.com>,
Vasily Gorbik <gor@...ux.ibm.com>,
linuxppc-dev@...ts.ozlabs.org, Stafford Horne <shorne@...il.com>,
"David S . Miller" <davem@...emloft.net>,
Johannes Berg <johannes@...solutions.net>,
Brian Cain <bcain@...cinc.com>, x86@...nel.org,
linux-parisc@...r.kernel.org, Richard Henderson <rth@...ddle.net>,
Christian Borntraeger <borntraeger@...ux.ibm.com>,
Richard Weinberger <richard@....at>,
linux-hexagon@...r.kernel.org,
Benjamin Herrenschmidt <benh@...nel.crashing.org>,
Thomas Gleixner <tglx@...utronix.de>,
Janosch Frank <frankja@...ux.ibm.com>,
Albert Ou <aou@...s.berkeley.edu>,
Anton Ivanov <anton.ivanov@...bridgegreys.com>,
Dave Hansen <dave.hansen@...ux.intel.com>,
linux-ia64@...r.kernel.org, Borislav Petkov <bp@...en8.de>,
Sven Schnelle <svens@...ux.ibm.com>,
Andrea Arcangeli <aarcange@...hat.com>,
"James E . J . Bottomley" <James.Bottomley@...senpartnership.com>,
Al Viro <viro@...iv.linux.org.uk>,
Alexander Gordeev <agordeev@...ux.ibm.com>,
Peter Zijlstra <peterz@...radead.org>,
Alistair Popple <apopple@...dia.com>,
Jonas Bonn <jonas@...thpole.se>, sparclinux@...r.kernel.org,
linux-csky@...r.kernel.org, Will Deacon <will@...nel.org>,
Andrew Morton <akpm@...ux-foundation.org>,
Vlastimil Babka <vbabka@...e.cz>, linux-um@...ts.infradead.org,
Michal Simek <monstr@...str.eu>,
Matt Turner <mattst88@...il.com>,
linux-m68k@...ts.linux-m68k.org, Paul Mackerras <paulus@...ba.org>,
linux-xtensa@...ux-xtensa.org,
Geert Uytterhoeven <geert@...ux-m68k.org>,
David Hildenbrand <david@...hat.com>,
openrisc@...ts.librecores.org,
linux-arm-kernel@...ts.infradead.org,
Nicholas Piggin <npiggin@...il.com>,
Palmer Dabbelt <palmer@...belt.com>,
Michael Ellerman <mpe@...erman.id.au>,
Stefan Kristiansson <stefan.kristiansson@...nalahti.fi>,
Paul Walmsley <paul.walmsley@...ive.com>,
Ivan Kokshaysky <ink@...assic.park.msu.ru>,
Chris Zankel <chris@...kel.net>,
Hugh Dickins <hughd@...gle.com>,
Vineet Gupta <vgupta@...nel.org>,
Dinh Nguyen <dinguyen@...nel.org>,
Catalin Marinas <catalin.marinas@....com>,
Rich Felker <dalias@...c.org>,
"H . Peter Anvin" <hpa@...or.com>, linux-s390@...r.kernel.org,
linux-sh@...r.kernel.org, linux-riscv@...ts.infradead.org,
Heiko Carstens <hca@...ux.ibm.com>,
Johannes Weiner <hannes@...xchg.org>,
Andy Lutomirski <luto@...nel.org>,
Max Filippov <jcmvbkbc@...il.com>, Guo Ren <guoren@...nel.org>,
Thomas Bogendoerfer <tsbogend@...ha.franken.de>,
linux-snps-arc@...ts.infradead.org, linux-alpha@...r.kernel.org,
linux-mips@...r.kernel.org, Helge Deller <deller@....de>,
Yoshinori Sato <ysato@...rs.sourceforge.jp>,
Ingo Molnar <mingo@...nel.org>
Subject: Re: [PATCH v5] mm: Avoid unnecessary page fault retires on shared
memory types
On Mon, May 30, 2022 at 02:34:50PM -0400, Peter Xu wrote:
> I observed that for each of the shared file-backed page faults, we're very
> likely to retry one more time for the 1st write fault upon no page. It's
> because we'll need to release the mmap lock for dirty rate limit purpose
> with balance_dirty_pages_ratelimited() (in fault_dirty_shared_page()).
>
> Then after that throttling we return VM_FAULT_RETRY.
>
> We did that probably because VM_FAULT_RETRY is the only way we can return
> to the fault handler at that time telling it we've released the mmap lock.
>
> However that's not ideal because it's very likely the fault does not need
> to be retried at all since the pgtable was well installed before the
> throttling, so the next continuous fault (including taking mmap read lock,
> walk the pgtable, etc.) could be in most cases unnecessary.
>
> It's not only slowing down page faults for shared file-backed, but also add
> more mmap lock contention which is in most cases not needed at all.
>
> To observe this, one could try to write to some shmem page and look at
> "pgfault" value in /proc/vmstat, then we should expect 2 counts for each
> shmem write simply because we retried, and vm event "pgfault" will capture
> that.
>
> To make it more efficient, add a new VM_FAULT_COMPLETED return code just to
> show that we've completed the whole fault and released the lock. It's also
> a hint that we should very possibly not need another fault immediately on
> this page because we've just completed it.
>
> This patch provides a ~12% perf boost on my aarch64 test VM with a simple
> program sequentially dirtying 400MB shmem file being mmap()ed and these are
> the time it needs:
>
> Before: 650.980 ms (+-1.94%)
> After: 569.396 ms (+-1.38%)
>
> I believe it could help more than that.
>
> We need some special care on GUP and the s390 pgfault handler (for gmap
> code before returning from pgfault), the rest changes in the page fault
> handlers should be relatively straightforward.
>
> Another thing to mention is that mm_account_fault() does take this new
> fault as a generic fault to be accounted, unlike VM_FAULT_RETRY.
>
> I explicitly didn't touch hmm_vma_fault() and break_ksm() because they do
> not handle VM_FAULT_RETRY even with existing code, so I'm literally keeping
> them as-is.
>
> Acked-by: Geert Uytterhoeven <geert@...ux-m68k.org>
> Acked-by: Peter Zijlstra (Intel) <peterz@...radead.org>
> Acked-by: Johannes Weiner <hannes@...xchg.org>
> Acked-by: Vineet Gupta <vgupta@...nel.org>
> Acked-by: Guo Ren <guoren@...nel.org>
> Acked-by: Max Filippov <jcmvbkbc@...il.com>
> Acked-by: Christian Borntraeger <borntraeger@...ux.ibm.com>
> Acked-by: Michael Ellerman <mpe@...erman.id.au> (powerpc)
> Acked-by: Catalin Marinas <catalin.marinas@....com>
> Reviewed-by: Alistair Popple <apopple@...dia.com>
> Reviewed-by: Ingo Molnar <mingo@...nel.org>
> Signed-off-by: Peter Xu <peterx@...hat.com>
For:
> diff --git a/arch/arm/mm/fault.c b/arch/arm/mm/fault.c
> index a062e07516dd..46cccd6bf705 100644
> --- a/arch/arm/mm/fault.c
> +++ b/arch/arm/mm/fault.c
> @@ -322,6 +322,10 @@ do_page_fault(unsigned long addr, unsigned int fsr, struct pt_regs *regs)
> return 0;
> }
>
> + /* The fault is fully completed (including releasing mmap lock) */
> + if (fault & VM_FAULT_COMPLETED)
> + return 0;
> +
> if (!(fault & VM_FAULT_ERROR)) {
> if (fault & VM_FAULT_RETRY) {
> flags |= FAULT_FLAG_TRIED;
Acked-by: Russell King (Oracle) <rmk+kernel@...linux.org.uk>
Thanks!
--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 40Mbps down 10Mbps up. Decent connectivity at last!
Powered by blists - more mailing lists