Message-ID: <20250409162837.GN1778492@nvidia.com>
Date: Wed, 9 Apr 2025 13:28:37 -0300
From: Jason Gunthorpe <jgg@...dia.com>
To: Mike Rapoport <rppt@...nel.org>
Cc: Pratyush Yadav <ptyadav@...zon.de>,
	Changyuan Lyu <changyuanl@...gle.com>, linux-kernel@...r.kernel.org,
	graf@...zon.com, akpm@...ux-foundation.org, luto@...nel.org,
	anthony.yznaga@...cle.com, arnd@...db.de, ashish.kalra@....com,
	benh@...nel.crashing.org, bp@...en8.de, catalin.marinas@....com,
	dave.hansen@...ux.intel.com, dwmw2@...radead.org,
	ebiederm@...ssion.com, mingo@...hat.com, jgowans@...zon.com,
	corbet@....net, krzk@...nel.org, mark.rutland@....com,
	pbonzini@...hat.com, pasha.tatashin@...een.com, hpa@...or.com,
	peterz@...radead.org, robh+dt@...nel.org, robh@...nel.org,
	saravanak@...gle.com, skinsburskii@...ux.microsoft.com,
	rostedt@...dmis.org, tglx@...utronix.de, thomas.lendacky@....com,
	usama.arif@...edance.com, will@...nel.org,
	devicetree@...r.kernel.org, kexec@...ts.infradead.org,
	linux-arm-kernel@...ts.infradead.org, linux-doc@...r.kernel.org,
	linux-mm@...ck.org, x86@...nel.org
Subject: Re: [PATCH v5 09/16] kexec: enable KHO support for memory
 preservation

On Wed, Apr 09, 2025 at 07:19:30PM +0300, Mike Rapoport wrote:
> On Wed, Apr 09, 2025 at 12:37:14PM -0300, Jason Gunthorpe wrote:
> > On Wed, Apr 09, 2025 at 04:58:16PM +0300, Mike Rapoport wrote:
> > > >
> > > > I think we still don't really know what will be needed, so I'd stick
> > > > with folio only as that allows building the memfd and a potential slab
> > > > preservation system.
> > > 
> > > void * seems to me much more reasonable than folio one as the starting
> > > point because it allows preserving folios with the right order but it's not
> > > limited to it. 
> > 
> > It would just call kho_preserve_folio() under the covers though.
> 
> How that will work for memblock and 1G pages?

memblock has to do its own thing, it isn't the buddy allocator.

1G pages should be very high order folios

> > It does for vmalloc too, just stop thinking about it as a
> > folio-for-pagecache and instead as an arbitary order handle to buddy
> > allocator memory that will someday be changed to a memdesc :|
> 
> But we have memdesc today, it's struct page.

No, I don't think it is. struct page seems to be turning into
something legacy that indicates the code has not been converted to the
new stuff yet.

> And when the data structure that memdesc points to will be allocated
> separately folios won't make sense for order-0 allocations.

At that point the lowest level allocator function will be allocating
the memdesc along with the struct page. Then folio will become
restricted to only actual folio memdescs and alot of the type punning
should go away. We are not there yet.

> > The lowest allocator primitive returns folios, which can represent any
> > order, and the caller casts to their own memdesc.
> 
> The lowest allocation primitive returns pages. 

Yes, but as I understand things, we should not be calling that
interface in new code because we are trying to make 'struct page' go
away.

Instead you should use the folio interfaces and cast to your own
memdesc, or use an allocator interface that returns void * (ie slab)
and never touch the struct page area.

AFAICT, and I just wrote one of these..

> And I don't think folio will be a lowest primitive buddy returns anytime
> soon if ever.

Maybe not internally, but driver facing, I think it should be true.

Like I just completely purged all struct page from the iommu code:

https://lore.kernel.org/linux-iommu/0-v4-c8663abbb606+3f7-iommu_pages_jgg@nvidia.com/

I don't want some weird KHO interface that doesn't align with using
__folio_alloc_node() and folio_put() as the lowest level allocator
interface.

Jason

Powered by blists - more mailing lists