netdev - Re: [PATCH vhost v11 05/10] virtio_ring: introduce virtqueue_dma

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Tue, 1 Aug 2023 08:45:10 -0700
From: Jakub Kicinski <kuba@...nel.org>
To: Xuan Zhuo <xuanzhuo@...ux.alibaba.com>
Cc: Christoph Hellwig <hch@...radead.org>,
 virtualization@...ts.linux-foundation.org, "David S.  Miller"
 <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>, Paolo  Abeni
 <pabeni@...hat.com>, Alexei Starovoitov <ast@...nel.org>, Daniel  Borkmann
 <daniel@...earbox.net>, Jesper Dangaard Brouer <hawk@...nel.org>, John 
 Fastabend <john.fastabend@...il.com>, netdev@...r.kernel.org,
 bpf@...r.kernel.org, "Michael S. Tsirkin" <mst@...hat.com>, Jason Wang
 <jasowang@...hat.com>, Pavel Begunkov <asml.silence@...il.com>
Subject: Re: [PATCH vhost v11 05/10] virtio_ring: introduce
 virtqueue_dma_dev()

On Tue, 1 Aug 2023 10:57:30 +0800 Xuan Zhuo wrote:
> > You have this working and benchmarked or this is just and idea?  
> 
> This is not just an idea. I said that has been used on large scale.
> 
> This is the library for the APP to use the AF_XDP. We has open it.
> https://gitee.com/anolis/libxudp
> 
> This is the Alibaba version of the nginx. That has been opened, that supported
> to work with the libray to use AF_XDP.
> http://tengine.taobao.org/
> 
> I supported this on our kernel release Anolis/Alinux.

Interesting!

> The work was done about 2 years ago. You know, I pushed the first version to
> enable AF_XDP on virtio-net about two years ago. I never thought the job would
> be so difficult.

Me neither, but it is what it is.

> The nic (virtio-net) of AliYun can reach 24,000,000PPS.
> So I think there is no different with the real HW on the performance.
> 
> With the AF_XDP, the UDP pps is seven times that of the kernel udp stack.

UDP pps or QUIC pps? UDP with or without GSO?

Do you have measurements of how much it saves in real world workloads?
I'm asking mostly out of curiosity, not to question the use case.

> > What about io_uring zero copy w/ pre-registered buffers.
> > You'll get csum offload, GSO, all the normal perf features.  
> 
> We tried io-uring, but it was not suitable for our scenario.
> 
> Yes, now the AF_XDP does not support the csum offload and GSO.
> This is indeed a small problem.

Can you say more about io-uring suitability? It can do zero copy
and recently-ish Pavel optimized it quite a bit.