linux-kernel - Re: Report: Performance regression from ib_umem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20250424120143.GX1213339@ziepe.ca>
Date: Thu, 24 Apr 2025 09:01:43 -0300
From: Jason Gunthorpe <jgg@...pe.ca>
To: jane.chu@...cle.com
Cc: logane@...tatee.com, hch@....de, gregkh@...uxfoundation.org,
	willy@...radead.org, kch@...dia.com, axboe@...nel.dk,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	linux-pci@...r.kernel.org, linux-nvme@...ts.infradead.org,
	linux-block@...r.kernel.org
Subject: Re: Report: Performance regression from ib_umem_get on zone device
 pages

On Wed, Apr 23, 2025 at 10:35:06PM -0700, jane.chu@...cle.com wrote:
> 
> On 4/23/2025 4:28 PM, Jason Gunthorpe wrote:
> > > The flow of a single test run:
> > >    1. reserve virtual address space for (61440 * 2MB) via mmap with PROT_NONE
> > > and MAP_ANONYMOUS | MAP_NORESERVE| MAP_PRIVATE
> > >    2. mmap ((61440 * 2MB) / 12) from each of the 12 device-dax to the
> > > reserved virtual address space sequentially to form a continual VA
> > > space
> > Like is there any chance that each of these 61440 VMA's is a single
> > 2MB folio from device-dax, or could it be?
> > 
> > IIRC device-dax does could not use folios until 6.15 so I'm assuming
> > it is not folios even if it is a pmd mapping?
> 
> I just ran the mr registration stress test in 6.15-rc3, much better!
> 
> What's changed?  is it folio for device-dax?  none of the code in
> ib_umem_get() has changed though, it still loops through 'npages' doing

I don't know, it is kind of strange that it changed. If device-dax is
now using folios then it does change the access pattern to the struct
page array somewhat, especially it moves all the writes to the head
page of the 2MB section which maybe impacts the the caching?

Jason