netdev - Re: [PATCH net-next] xsk: introduce xsk_dma

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1681911643.4417202-1-xuanzhuo@linux.alibaba.com>
Date:   Wed, 19 Apr 2023 21:40:43 +0800
From:   Xuan Zhuo <xuanzhuo@...ux.alibaba.com>
To:     Alexander Lobakin <aleksander.lobakin@...el.com>
Cc:     <netdev@...r.kernel.org>,
        Björn Töpel <bjorn@...nel.org>,
        Magnus Karlsson <magnus.karlsson@...el.com>,
        Maciej Fijalkowski <maciej.fijalkowski@...el.com>,
        Jonathan Lemon <jonathan.lemon@...il.com>,
        "David S. Miller" <davem@...emloft.net>,
        Eric Dumazet <edumazet@...gle.com>,
        Paolo Abeni <pabeni@...hat.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Jesper Dangaard Brouer <hawk@...nel.org>,
        John Fastabend <john.fastabend@...il.com>,
        <bpf@...r.kernel.org>, <virtualization@...ts.linux-foundation.org>,
        "Michael S. Tsirkin" <mst@...hat.com>,
        Guenter Roeck <linux@...ck-us.net>,
        Gerd Hoffmann <kraxel@...hat.com>,
        Jason Wang <jasowang@...hat.com>,
        "Greg Kroah-Hartman" <gregkh@...uxfoundation.org>,
        Jens Axboe <axboe@...nel.dk>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Christoph Hellwig <hch@...radead.org>,
        Jakub Kicinski <kuba@...nel.org>
Subject: Re: [PATCH net-next] xsk: introduce xsk_dma_ops

On Wed, 19 Apr 2023 15:14:48 +0200, Alexander Lobakin <aleksander.lobakin@...el.com> wrote:
> From: Christoph Hellwig <hch@...radead.org>
> Date: Tue, 18 Apr 2023 22:16:53 -0700
>
> > On Mon, Apr 17, 2023 at 11:19:47PM -0700, Jakub Kicinski wrote:
> >>> You can't just do dma mapping outside the driver, because there are
> >>> drivers that do not require DMA mapping at all.  virtio is an example,
> >>> but all the classic s390 drivers and some other odd virtualization
> >>> ones are others.
> >>
> >> What bus are the classic s390 on (in terms of the device model)?
> >
> > I think most of them are based on struct ccw_device, but I'll let the
> > s390 maintainers fill in.
> >
> > Another interesting case that isn't really relevant for your networking
> > guys, but that caused as problems is RDMA.  For hardware RDMA devices
> > it wants the ULPs to DMA map, but it turns out we have various software
> > drivers that map to network drivers that do their own DMA mapping
> > at a much lower layer and after potentially splitting packets or
> > even mangling them.
> >
> >>
> >>>> I don't think it's reasonable to be bubbling up custom per-subsystem
> >>>> DMA ops into all of them for the sake of virtio.
> >>>
> >>> dma addresses and thus dma mappings are completely driver specific.
> >>> Upper layers have no business looking at them.
>
> Here it's not an "upper layer". XSk core doesn't look at them or pass
> them between several drivers. It maps DMA solely via the struct device
> passed from the driver and then just gets-sets addresses for this driver
> only. Just like Page Pool does for regular Rx buffers. This got moved to
> the XSk core to not repeat the same code pattern in each driver.
>
> >>
> >> Damn, that's unfortunate. Thinking aloud -- that means that if we want
> >> to continue to pull memory management out of networking drivers to
> >> improve it for all, cross-optimize with the rest of the stack and
> >> allow various upcoming forms of zero copy -- then we need to add an
> >> equivalent of dma_ops and DMA API locally in networking?
>
> Managing DMA addresses is totally fine as long as you don't try to pass
> mapped addresses between different drivers :D Page Pool already does
> that and I don't see a problem in that in general.
>
> >
> > Can you explain what the actual use case is?
> >
> >>From the original patchset I suspect it is dma mapping something very
> > long term and then maybe doing syncs on it as needed?
>
> As I mentioned, XSk provides some handy wrappers to map DMA for drivers.
> Previously, XSk was supported by real hardware drivers only, but here
> the developer tries to add support to virtio-net. I suspect he needs to
> use DMA mapping functions different from which the regular driver use.
> So this is far from dma_map_ops, the author picked wrong name :D


Yes, dma_ops caused some misunderstandings, which is indeed a wrong name.

Thanks.


> And correct, for XSk we map one big piece of memory only once and then
> reuse it for buffers, no inflight map/unmap on hotpath (only syncs when
> needed). So this mapping is longterm and is stored in XSk core structure
> assigned to the driver which this mapping was done for.
> I think Jakub thinks of something similar, but for the "regular" Rx/Tx,
> not only XDP sockets :)
>
> Thanks,
> Olek