netdev - Re: [PATCH 5/5] io_uring/notif: let userspace know how effective the zero copy usage was

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <707e5141-aeeb-f54d-46d4-6a7575bfbc4d@samba.org>
Date:   Wed, 21 Sep 2022 14:33:22 +0200
From:   Stefan Metzmacher <metze@...ba.org>
To:     Pavel Begunkov <asml.silence@...il.com>, io-uring@...r.kernel.org,
        axboe@...nel.dk
Cc:     Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org
Subject: Re: [PATCH 5/5] io_uring/notif: let userspace know how effective the
 zero copy usage was

Am 21.09.22 um 14:04 schrieb Pavel Begunkov:
> On 9/17/22 11:24, Stefan Metzmacher wrote:
>> Am 17.09.22 um 11:22 schrieb Pavel Begunkov:
>>> On 9/16/22 22:36, Stefan Metzmacher wrote:
>>>> The 2nd cqe for IORING_OP_SEND_ZC has IORING_CQE_F_NOTIF set in cqe->flags
>>>> and it will now have the number of successful completed
>>>> io_uring_tx_zerocopy_callback() callbacks in the lower 31-bits
>>>> of cqe->res, the high bit (0x80000000) is set when
>>>> io_uring_tx_zerocopy_callback() was called with success=false.
>>>
>>> It has a couple of problems, and because that "simplify uapi"
>>> patch is transitional it doesn't go well with what I'm queuing
>>> for 6.1, let's hold it for a while.
>>
>> Once the current behavior gets released stable, we're no
>> longer able to change the meaning of cqe.res.
>>
>> As cqe.res == 0 would mean zero copy wasn't used at all,
>> which would be the indication for userspace to avoid using SEND_ZC.
>>
>> But if 6.0 would always return cqe.res = 0, there's no chance for
>> userspace to have a detection strategy.
>>
>> And I don't think it will cause a lot of trouble for your 6.1 stuff (assuming
>> you mean your SENDMSG_ZC code), I was already having that on top
>> of my test branches, the current one is:
>> https://git.samba.org/?p=metze/linux/wip.git;a=shortlog;h=refs/heads/io_uring-6.0.0-rc5-metze.08
> 
> Not that one though, 1) I want to shrink ubuf_info as we're a bit out
> of space on the io_uring side and it prevents some embedding, so there
> won't be additional fields you used. And 2) we want to have a feature
> merging completion + notif CQEs into one (useful for UDP and some TCP
> cases), but that would mean we'll be using cqe->res for the normal
> return value.

But wouldn't that just don't have the MORE flag set, and be just
like IO_SEND?

> We can disable the success counting in this case, but it's not nice,
> and we can't actually count in io_uring_tx_zerocopy_callback() as in
> the patch (racy). Also, the callback will be called multiple times for
> a number of different reasons like io_uring flush or net splitting skbs.
> The number won't be much useful and unnecessary exposes internal details,
> so I think F_COPIED in cqe->flags is a better option.

Ok, that's better than nothing.

> It's a good question though whether we need more versatile reporting and
> how to do it right. Probably should be optional and not a part of IO path,
> e.g. send(MSG_PROBE) / ioctl / proc stats / etc.
> 
>> I plan to test SENDMSG_ZC with Samba next week.
> 
> Awesome

Currently I'm fighting with converting samba's libtevent to
use POLL_ADD and io_uring_submit_and_wait_timeout().
Which turns out to be much more complex that it should be.
And IORING_POLL_ADD_LEVEL doesn't seem to work at all...
But I'll write a separate mail about my findings in that area...

metze