netdev - Re: [PATCH net-next v6 08/12] net: homa: create homa

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAGXJAmxDozdg3FPDNkFUcQU9FXENr-Oefnp61eWzXo5Sne4C1g@mail.gmail.com>
Date: Fri, 31 Jan 2025 14:51:13 -0800
From: John Ousterhout <ouster@...stanford.edu>
To: Paolo Abeni <pabeni@...hat.com>
Cc: netdev@...r.kernel.org, edumazet@...gle.com, horms@...nel.org, 
	kuba@...nel.org
Subject: Re: [PATCH net-next v6 08/12] net: homa: create homa_incoming.c

Also resending this message to get rid of HTML in the original...

On Thu, Jan 30, 2025 at 1:57 AM Paolo Abeni <pabeni@...hat.com> wrote:
> On 1/30/25 1:48 AM, John Ousterhout wrote:
> > On Mon, Jan 27, 2025 at 2:19 AM Paolo Abeni <pabeni@...hat.com> wrote:
> >>
> >> On 1/15/25 7:59 PM, John Ousterhout wrote:
> >>> +     /* Each iteration through the following loop processes one packet. */
> >>> +     for (; skb; skb = next) {
> >>> +             h = (struct homa_data_hdr *)skb->data;
> >>> +             next = skb->next;
> >>> +
> >>> +             /* Relinquish the RPC lock temporarily if it's needed
> >>> +              * elsewhere.
> >>> +              */
> >>> +             if (rpc) {
> >>> +                     int flags = atomic_read(&rpc->flags);
> >>> +
> >>> +                     if (flags & APP_NEEDS_LOCK) {
> >>> +                             homa_rpc_unlock(rpc);
> >>> +                             homa_spin(200);
> >>
> >> Why spinning on the current CPU here? This is completely unexpected, and
> >> usually tolerated only to deal with H/W imposed delay while programming
> >> some device registers.
> >
> > This is done to pass the RPC lock off to another thread (the
> > application); the spin is there to allow the other thread to acquire
> > the lock before this thread tries to acquire it again (almost
> > immediately). There's no performance impact from the spin because this
> > thread is going to turn around and try to acquire the RPC lock again
> > (at which point it will spin until the other thread releases the
> > lock). Thus it's either spin here or spin there. I've added a comment
> > to explain this.
>
> What if another process is spinning on the RPC lock without setting
> APP_NEEDS_LOCK? AFAICS incoming packets targeting the same RPC could
> land on different RX queues.

If that happens then it could grab the lock instead of the desired
application, which would defeat the performance optimization and delay
the application a bit. This would be no worse than if the
APP_NEEDS_LOCK mechanism were not present.

> If the spin is not functionally needed, just drop it. If it's needed, it
> would be better to find some functional replacement, possibly explicit
> notification via waitqueue or completion.

The goal is to have a very lightweight mechanism for an application to
preempt the RPC lock. I'd be happy to use an existing mechanism if
something appropriate exists, but waitqueues and completions sound
more heavyweight to me; aren't they both based on blocking rather than
spinning?

One of the reasons Homa has rolled its own mechanisms is that it's
trying to operate at a timescale that's different from the rest of the
kernel.

-John-