[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAHS8izMkCwv3jak9KUHeDUrkwBNNpdYk4voEX7Cbp7mTpNAQdA@mail.gmail.com>
Date: Tue, 19 Dec 2023 19:01:14 -0800
From: Mina Almasry <almasrymina@...gle.com>
To: Shakeel Butt <shakeelb@...gle.com>
Cc: Jakub Kicinski <kuba@...nel.org>, Yunsheng Lin <linyunsheng@...wei.com>,
linux-kernel@...r.kernel.org, netdev@...r.kernel.org, bpf@...r.kernel.org,
Thomas Gleixner <tglx@...utronix.de>, Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
Dave Hansen <dave.hansen@...ux.intel.com>, x86@...nel.org,
"H. Peter Anvin" <hpa@...or.com>, Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
"Rafael J. Wysocki" <rafael@...nel.org>, Sumit Semwal <sumit.semwal@...aro.org>,
Christian König <christian.koenig@....com>,
Michael Chan <michael.chan@...adcom.com>, "David S. Miller" <davem@...emloft.net>,
Eric Dumazet <edumazet@...gle.com>, Paolo Abeni <pabeni@...hat.com>,
Alexei Starovoitov <ast@...nel.org>, Daniel Borkmann <daniel@...earbox.net>,
Jesper Dangaard Brouer <hawk@...nel.org>, John Fastabend <john.fastabend@...il.com>, Wei Fang <wei.fang@....com>,
Shenwei Wang <shenwei.wang@....com>, Clark Wang <xiaoning.wang@....com>,
NXP Linux Team <linux-imx@....com>, Jeroen de Borst <jeroendb@...gle.com>,
Praveen Kaligineedi <pkaligineedi@...gle.com>, Shailend Chand <shailend@...gle.com>,
Yisen Zhuang <yisen.zhuang@...wei.com>, Salil Mehta <salil.mehta@...wei.com>,
Jesse Brandeburg <jesse.brandeburg@...el.com>, Tony Nguyen <anthony.l.nguyen@...el.com>,
Thomas Petazzoni <thomas.petazzoni@...tlin.com>, Marcin Wojtas <mw@...ihalf.com>,
Russell King <linux@...linux.org.uk>, Sunil Goutham <sgoutham@...vell.com>,
Geetha sowjanya <gakula@...vell.com>, Subbaraya Sundeep <sbhatta@...vell.com>,
hariprasad <hkelam@...vell.com>, Felix Fietkau <nbd@....name>, John Crispin <john@...ozen.org>,
Sean Wang <sean.wang@...iatek.com>, Mark Lee <Mark-MC.Lee@...iatek.com>,
Lorenzo Bianconi <lorenzo@...nel.org>, Matthias Brugger <matthias.bgg@...il.com>,
AngeloGioacchino Del Regno <angelogioacchino.delregno@...labora.com>,
Saeed Mahameed <saeedm@...dia.com>, Leon Romanovsky <leon@...nel.org>,
Horatiu Vultur <horatiu.vultur@...rochip.com>, UNGLinuxDriver@...rochip.com,
"K. Y. Srinivasan" <kys@...rosoft.com>, Haiyang Zhang <haiyangz@...rosoft.com>, Wei Liu <wei.liu@...nel.org>,
Dexuan Cui <decui@...rosoft.com>, Jassi Brar <jaswinder.singh@...aro.org>,
Ilias Apalodimas <ilias.apalodimas@...aro.org>,
Alexandre Torgue <alexandre.torgue@...s.st.com>, Jose Abreu <joabreu@...opsys.com>,
Maxime Coquelin <mcoquelin.stm32@...il.com>, Siddharth Vadapalli <s-vadapalli@...com>,
Ravi Gunasekaran <r-gunasekaran@...com>, Roger Quadros <rogerq@...nel.org>,
Jiawen Wu <jiawenwu@...stnetic.com>, Mengyuan Lou <mengyuanlou@...-swift.com>,
Ronak Doshi <doshir@...are.com>, VMware PV-Drivers Reviewers <pv-drivers@...are.com>,
Ryder Lee <ryder.lee@...iatek.com>, Shayne Chen <shayne.chen@...iatek.com>,
Kalle Valo <kvalo@...nel.org>, Juergen Gross <jgross@...e.com>,
Stefano Stabellini <sstabellini@...nel.org>,
Oleksandr Tyshchenko <oleksandr_tyshchenko@...m.com>, Andrii Nakryiko <andrii@...nel.org>,
Martin KaFai Lau <martin.lau@...ux.dev>, Song Liu <song@...nel.org>,
Yonghong Song <yonghong.song@...ux.dev>, KP Singh <kpsingh@...nel.org>,
Stanislav Fomichev <sdf@...gle.com>, Hao Luo <haoluo@...gle.com>, Jiri Olsa <jolsa@...nel.org>,
Stefan Hajnoczi <stefanha@...hat.com>, Stefano Garzarella <sgarzare@...hat.com>, Shuah Khan <shuah@...nel.org>,
Mickaël Salaün <mic@...ikod.net>,
Nathan Chancellor <nathan@...nel.org>, Nick Desaulniers <ndesaulniers@...gle.com>,
Bill Wendling <morbo@...gle.com>, Justin Stitt <justinstitt@...gle.com>,
Jason Gunthorpe <jgg@...dia.com>, Willem de Bruijn <willemdebruijn.kernel@...il.com>
Subject: Re: [RFC PATCH net-next v1 4/4] net: page_pool: use netmem_t instead
of struct page in API
On Sat, Dec 16, 2023 at 2:06 PM Mina Almasry <almasrymina@...gle.com> wrote:
>
> On Sat, Dec 16, 2023 at 11:47 AM Shakeel Butt <shakeelb@...gle.com> wrote:
> >
> > On Fri, Dec 15, 2023 at 7:01 PM Jakub Kicinski <kuba@...nel.org> wrote:
> > >
> > > On Fri, 15 Dec 2023 02:11:14 +0000 Shakeel Butt wrote:
> > > > > From my POV it has to be the first one. We want to abstract the memory
> > > > > type from the drivers as much as possible, not introduce N new memory
> > > > > types and ask the driver to implement new code for each of them
> > > > > separately.
> > > >
> > > > Agree with Mina's point. Let's aim to decouple memory types from
> > > > drivers.
> > >
> > > What does "decouple" mean? Drivers should never convert netmem
> > > to pages. Either a path in the driver can deal with netmem,
> > > i.e. never touch the payload, or it needs pages.
> >
>
> I'm guessing the paths in the driver that need pages will have to be
> disabled for non-paged netmem, which is fine.
>
> One example that I ran into with GVE is that it calls page_address()
> to copy small packets instead of adding them as a frag. I can add a
> netmem_address() that returns page_address() for pages, and NULL for
> non-pages (never passing non-pages to mm code). The driver can detect
> that the netmem has no address, and disable the optimization for
> non-paged netmem.
>
> > "Decouple" might not be the right word. What I wanted to say was to
> > avoid too much specialization such that we have to have a new API for
> > every new fancy thing.
> >
> > >
> > > Perhaps we should aim to not export netmem_to_page(),
> > > prevent modules from accessing it directly.
> >
> > +1.
>
I looked into this, but it turns out it's a slightly bigger change
that needs some refactoring to make it work. There are few places
where I believe I need to add netmem_to_page() that are exposed to the
drivers via inline helpers, these are:
- skb_frag_page(), which returns NULL if the netmem is not a page, but
needs to do a netmem_to_page() to return the page otherwise.
- The helpers inside skb_add_rx_frag(), which needs to do a
netmem_to_page() to set skb->pfmemalloc.
- Some of the page_pool APIs are exposed to the drivers as static
inline helpers, and if I want the page_pool to use netmem internally
the page_pool needs to do a netmem_to_page() in these helpers.
The refactor is not an issue, but I was wondering if not exporting
netmem_to_page() was worth moving the code around. I was thinking in
the interim until netmem is adopted and has actual driver users we may
prefer to just add a comment on the netmem_to_page() helper that says
'try not to use this directly and use the netmem helpers instead'.
> This is an aggressive approach and I like it. I'll try to make it work
> (should be fine).
>
>
> --
> Thanks,
> Mina
--
Thanks,
Mina
Powered by blists - more mailing lists