netdev - Re: Upstream Homa?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y3FVpQAWI7kTzfhf@lunn.ch>
Date:   Sun, 13 Nov 2022 21:37:57 +0100
From:   Andrew Lunn <andrew@...n.ch>
To:     John Ousterhout <ouster@...stanford.edu>
Cc:     Jiri Pirko <jiri@...nulli.us>,
        Stephen Hemminger <stephen@...workplumber.org>,
        netdev@...r.kernel.org
Subject: Re: Upstream Homa?

On Sun, Nov 13, 2022 at 12:10:22PM -0800, John Ousterhout wrote:
> On Sun, Nov 13, 2022 at 9:10 AM Andrew Lunn <andrew@...n.ch> wrote:
> >
> > > Homa implements RPCs rather than streams like TCP or messages like
> > > UDP. An RPC consists of a request message sent from client to server,
> > > followed by a response message from server back to client. This requires
> > > additional information in the API beyond what is provided in the arguments to
> > > sendto and recvfrom. For example, when sending a request message, the
> > > kernel returns an RPC identifier back to the application; when waiting for
> > > a response, the application can specify that it wants to receive the reply for
> > > a specific RPC identifier (or, it can specify that it will accept any
> > > reply, or any
> > > request, or both).
> >
> > This sounds like the ancillary data you can pass to sendmsg(). I've
> > not checked the code, it might be the current plumbing is only into to
> > the kernel, but i don't see why you cannot extend it to also allow
> > data to be passed back to user space. If this is new functionality,
> > maybe add a new flags argument to control it.
> >
> > recvmsg() also has ancillary data.
> 
> Whoah! I'd never noticed the msg_control and msg_controllen fields before.
> These may be sufficient to do everything Homa needs. Thanks for pointing
> this out.

Is zero copy also required? https://lwn.net/Articles/726917/ talks
about this. But rather than doing the transmit complete notification
via MSG_ERRORQUEUE, maybe you could make it part of the ancillary data
for a later message? That could save you some system calls? Or is the
latency low enough that the RPC reply acts an implicitly indication
the transmit buffer can be recycled?

If your aim is to offload Homa to the NIC, it seems like zero copy is
something you want, so even if you are not implementing it now, you
probably should consider what the uAPI looks like.

	 Andrew