Message-ID: <aFGYr6qlzVlqhK55@harry>
Date: Wed, 18 Jun 2025 01:32:47 +0900
From: Harry Yoo <harry.yoo@...cle.com>
To: David Hildenbrand <david@...hat.com>
Cc: Mina Almasry <almasrymina@...gle.com>, willy@...radead.org,
        Byungchul Park <byungchul@...com>, Jakub Kicinski <kuba@...nel.org>,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org, kernel_team@...ynix.com,
        ilias.apalodimas@...aro.org, hawk@...nel.org,
        akpm@...ux-foundation.org, davem@...emloft.net,
        john.fastabend@...il.com, andrew+netdev@...n.ch,
        asml.silence@...il.com, toke@...hat.com, tariqt@...dia.com,
        edumazet@...gle.com, pabeni@...hat.com, saeedm@...dia.com,
        leon@...nel.org, ast@...nel.org, daniel@...earbox.net,
        lorenzo.stoakes@...cle.com, Liam.Howlett@...cle.com, vbabka@...e.cz,
        rppt@...nel.org, surenb@...gle.com, mhocko@...e.com, horms@...nel.org,
        linux-rdma@...r.kernel.org, bpf@...r.kernel.org,
        vishal.moola@...il.com
Subject: Re: netmem series needs some love and Acks from MM folks

On Tue, Jun 17, 2025 at 06:09:36PM +0200, David Hildenbrand wrote:
> On 17.06.25 04:31, Harry Yoo wrote:
> > On Fri, Jun 13, 2025 at 07:19:07PM -0700, Mina Almasry wrote:
> > > On Thu, Jun 12, 2025 at 6:13 PM Byungchul Park <byungchul@...com> wrote:
> > > > 
> > > > On Wed, Jun 11, 2025 at 06:55:42PM -0700, Jakub Kicinski wrote:
> > > > > On Tue, 10 Jun 2025 10:30:01 +0900 Byungchul Park wrote:
> > > > > > > What's the intended relation between the types?
> > > > > > 
> > > > > > One thing I'm trying to achieve is to remove pp fields from struct page,
> > > > > > and make network code use struct netmem_desc { pp fields; } instead of
> > > > > > sturc page for that purpose.
> > > > > > 
> > > > > > The reason why I union'ed it with the existing pp fields in struct
> > > > > > net_iov *temporarily* for now is, to fade out the existing pp fields
> > > > > > from struct net_iov so as to make the final form like:
> > > > > 
> > > > > I see, I may have mixed up the complaints there. I thought the effort
> > > > > was also about removing the need for the ref count. And Rx is
> > > > > relatively light on use of ref counting.
> > > > > 
> > > > > > > netmem_ref exists to clearly indicate that memory may not be readable.
> > > > > > > Majority of memory we expect to allocate from page pool must be
> > > > > > > kernel-readable. What's the plan for reading the "single pointer"
> > > > > > > memory within the kernel?
> > > > > > > 
> > > > > > > I think you're approaching this problem from the easiest and least
> > > > > > 
> > > > > > No, I've never looked for the easiest way.  My bad if there are a better
> > > > > > way to achieve it.  What would you recommend?
> > > > > 
> > > > > Sorry, I don't mean that the approach you took is the easiest way out.
> > > > > I meant that between Rx and Tx handling Rx is the easier part because
> > > > > we already have the suitable abstraction. It's true that we use more
> > > > > fields in page struct on Rx, but I thought Tx is also more urgent
> > > > > as there are open reports for networking taking references on slab
> > > > > pages.
> > > > > 
> > > > > In any case, please make sure you maintain clear separation between
> > > > > readable and unreadable memory in the code you produce.
> > > > 
> > > > Do you mean the current patches do not?  If yes, please point out one
> > > > as example, which would be helpful to extract action items.
> > > > 
> > > 
> > > I think one thing we could do to improve separation between readable
> > > (pages/netmem_desc) and unreadable (net_iov) is to remove the struct
> > > netmem_desc field inside the net_iov, and instead just duplicate the
> > > pp/pp_ref_count/etc fields. The current code gives off the impression
> > > that net_iov may be a container of netmem_desc which is not really
> > > accurate.
> > > 
> > > But I don't think that's a major blocker. I think maybe the real issue
> > > is that there are no reviews from any mm maintainers?
> > 
> > Let's try changing the subject to draw some attention from MM people :)
> 
> Hi, it worked! :P

Glad it worked :)

> > 
> > > So I'm not 100%
> > > sure this is in line with their memdesc plans. I think probably
> > > patches 2->8 are generic netmem-ifications that are good to merge
> > > anyway, but I would say patch 1 and 9 need a reviewed by from someone
> > > on the mm side. Just my 2 cents.
> > 
> > As someone who worked on the zpdesc series, I think it is pretty much
> > in line with the memdesc plans.
> > 
> > I mean, it does differ a bit from the initial idea of generalizing it as
> > "bump" allocator, but overall, it's still aligned with the memdesc
> > plans, and looks like a starting point, IMHO.
> 
> Just to summarize (not that there is any misunderstanding), the first step
> of the memdesc plan is simple:
> 
> 1) have a dedicated data-structure we will allocate alter dynamically.
> 
> 2) Make it overlay "struct page" for now in a way that doesn't break things
> 
> 3) Convert all users of "struct page" to the new data-structure
> 
> Later, the memdesc data-structure will then actually come be allocated
> dynamically, so "struct page" content will not apply anymore, and we can
> shrink "struct page".
> 
> 
> What I see in this patch is exactly 1) and 2).

I believe 3) will be accompolished with a follow-up patch series from
Byungchul. The previous series was 18 patches, but divided to two series
and this is the first half.

> I am not 100% sure about existing "struct net_iov" and how that interacts
> with "struct page" overlay. I suspects it's just a dynamically allocated
> structure?

Yeah, that's indeed confusing. As mentioned here [1], it does not
overlay struct page at all. It is dynamically allocated from slab.

IIUC net_iov mirrors netmem_desc fields because netmem_ref might be
netmem_desc (overlays struct page) or net_iov (allocated from slab), and
by mirroring netmem_desc fields in net_iov, page pool doesn't need to care
if it's net_iov or netmem_desc when acccessing the common fields shared
between them.

[1] https://lore.kernel.org/linux-mm/aFDQ9X2WsXszNJ5M@hyeyoo

> Because this patch changes the layout of "struct net_iov", which is a bit
> confusing at first sight?

Yeah, better documented.

-- 
Cheers,
Harry / Hyeonggon

Powered by blists - more mailing lists