Message-ID: <CAGXJAmxqk14LdwtxER24X4wuFO0SrjALabE3=t7SatugbLqOQg@mail.gmail.com>
Date: Mon, 23 Dec 2024 09:17:44 -0800
From: John Ousterhout <ouster@...stanford.edu>
To: Arnd Bergmann <arnd@...db.de>
Cc: Jakub Kicinski <kuba@...nel.org>, Netdev <netdev@...r.kernel.org>, 
	Paolo Abeni <pabeni@...hat.com>, Eric Dumazet <edumazet@...gle.com>, Simon Horman <horms@...nel.org>
Subject: Re: [PATCH net-next v4 01/12] inet: homa: define user-visible API for Homa

On Sat, Dec 21, 2024 at 5:43 AM Arnd Bergmann <arnd@...db.de> wrote:
>
> >> That probably also explains what type of memory the
> >> __user buffer can point to, but I would like to make
> >> sure that this has well-defined behavior e.g. if that
> >> buffer is an mmap()ed file on NFS that was itself
> >> mounted over a homa socket. Is there any guarantee that
> >> this is either prohibited or is free of deadlocks and
> >> recursion?
> >
> > Given the API incompatibilities between Homa and TCP, I don't think it
> > is possible to have NFS mounted over a Homa socket. But you raise the
> > issue of whether some kinds of addresses might not be suitable for
> > Homa's buffer use this way. I don't know enough about the various
> > possible kinds of memory to know what kinds of problems could occur.
> > My assumption is that the buffer area will be a simple mmap()ed
> > region. The only use Homa makes of the buffer address is to call
> > import_ubuf with addresses in the buffer region, followed by
> > skb_copy_datagram_iter with the resulting iov_iter.
>
> Right, NFS was just an example, but there are other interesting
> cases. You certainly have to deal with buffers in userspace
> memory that are blocked indefinitely. Another interesting case
> is memory that has additional constraints, e.g. the MMIO
> space of a PCI device like a GPU, which may fault when writing
> data into it, or which cannot be mapped into the DMA space
> of a network device.

Homa doesn't map the user receive buffers into DMA space, so this last
issue won't come up.

> > Is there some way I can check the "kind" of memory behind the buffer
> > pointer, so Homa could reject anything other than the simple case?
>
> I don't think so. I still don't know what the exact constraints
> are that you have here, but I suspect this would all be a lot
> simpler if you could change the interface to not pass arbitrary
> user addresses but instead have a single file descriptor that
> backs the buffers, either by passing a tmpfs/hugetlbfs file into
> the socket instead of a pointer, or by using mmap() on the
> socket to map it into userspace like we do for packet sockets.

After thinking about this some more, I don't think there should be any
need for Homa to constrain the addresses passed into it. Homa's usage
of buffer addresses is the same as TCP's; the only difference is that
instead of receiving a buffer address in a recvmsg call, the address
is passed in earlier in a setsockopt call. In either case, the
application could pass in an arbitrary address. Assuming that TCP
properly handles all of the possible variations in addresses that
could be passed in, Homa should properly handle them as well, since it
invokes the same underlying functions. Is there a specific example you
have in mind of an address that would be problematic for Homa but not
also problematic for TCP?

-John-

Powered by blists - more mailing lists