linux-kernel - Re: [PATCH net-next v4 00/27] io

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <24b050e0-433f-dc97-7aab-15c9175f49fa@gmail.com>
Date:   Mon, 26 Sep 2022 21:08:27 +0100
From:   Pavel Begunkov <asml.silence@...il.com>
To:     David Ahern <dsahern@...nel.org>, io-uring@...r.kernel.org,
        netdev@...r.kernel.org, linux-kernel@...r.kernel.org
Cc:     "David S . Miller" <davem@...emloft.net>,
        Jakub Kicinski <kuba@...nel.org>,
        Jonathan Lemon <jonathan.lemon@...il.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Jens Axboe <axboe@...nel.dk>, kernel-team@...com
Subject: Re: [PATCH net-next v4 00/27] io_uring zerocopy send

On 7/24/22 19:28, David Ahern wrote:
> On 7/17/22 8:19 PM, David Ahern wrote:
>>>
>>> Haven't seen it back then. In general io_uring doesn't stop submitting
>>> requests if one request fails, at least because we're trying to execute
>>> requests asynchronously. And in general, requests can get executed
>>> out of order, so most probably submitting a bunch of requests to a single
>>> TCP sock without any ordering on io_uring side is likely a bug.
>>
>> TCP socket buffer fills resulting in a partial send (i.e, for a given
>> sqe submission only part of the write/send succeeded). io_uring was not
>> handling that case.
>>
>> I'll try to find some time to resurrect the iperf3 patch and try top of
>> tree kernel.
> 
> With your zc_v5 branch (plus the init fix on using msg->sg_from_iter),
> iperf3 with io_uring support (non-ZC case) no longer shows completions
> with incomplete sends. So that is good improvement over the last time I
> tried it.
> 
> However, adding in the ZC support and that problem resurfaces - a lot of
> completions are for an incomplete size.
> 
> liburing comes from your tree, zc_v4 branch. Upstream does not have
> support for notifications yet, so I can not move to it.
> 
> Changes to iperf3 are here:
>     https://github.com/dsahern/iperf mods-3.10-io_uring

Tried it out, the branch below fixes a small problem, adds a couple
of extra optimisations and now it actually uses registered buffers.

     https://github.com/isilence/iperf iou-sendzc

Still, the submission loop looked a bit weird, i.e. it submits I/O
to io_uring only when it exhausts sqes instead of sending right
away with some notion of QD and/or sending in batches. The approach
is good for batching (SQ size =16 here), but not so for latency.

I also see some CPU cycles being burnt in select(2). io_uring wait
would be more natural and perhaps more performant, but I didn't
spend enough time with iperf to say for sure.

-- 
Pavel Begunkov