netdev - Re: Advice on upstreaming Homa

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <eea46d0f-4676-4393-940c-27d6fc984938@lunn.ch>
Date: Tue, 1 Oct 2024 16:35:47 +0200
From: Andrew Lunn <andrew@...n.ch>
To: John Ousterhout <ouster@...stanford.edu>
Cc: Przemek Kitszel <przemyslaw.kitszel@...el.com>, netdev@...r.kernel.org
Subject: Re: Advice on upstreaming Homa

> I think Homa is already pretty well known to the netdev community:
> I've given talks on it at multiple NetDev conferences and there are a
> couple of published papers describing both the overall protocol and
> the Linux implementation.There is also a Wiki with lots of links to
> topics related to Homa

I agree with that. Maybe cut/paste an executive summary into patch
0/X, and include some links.

> (https://homa-transport.atlassian.net/wiki/spaces/HOMA/overview). Are
> you suggesting that Homa needs RFC standardization before uploading?

No. The kernel has plenty of experimental features. But you need to be
aware of the backwards compatibility issues you might be opening
yourself up to. Going forward, you cannot break running systems by
changing the protocol without maintaining backwards compatibility. So
make sure you have version fields where you need them, so you can in
the future negotiate between a stone age HOMA and a brand spanking new
version.

> There are a couple of Homa GitHub repos, and I suspect you may be
> looking at the one that is implemented in user space using DPDK. The
> kernel module I'd like to upstream is in this repo:
> https://github.com/PlatformLab/HomaModule. This is an in-kernel
> implementation that doesn't use DPDK.

Great. Kernel people will push back on anything which is not actual
needed for in the kernel.

> 
> > > Homa contains about 15 Klines of code. I have heard conflicting
> > > suggestions about how much to break it up for the upstreaming process,
> > > ranging from "just do it all in one patch set" to "it will need to be
> > > chopped up into chunks of a few hundred lines". The all-at-once
> > > approach is certainly easiest for me, and if it's broken up, the
> > > upstreamed code won't be functional until a significant fraction of it
> > > has been upstreamed. What's the recommended approach here?
> >
> > the split up into patches is to have it easiest to review, test
> > incrementally, and so on
> 
> I would be happy to have Homa reviewed, but is that likely, given its
> size? In any case, Homa has pretty extensive unit tests already.

It will get reviewed, but maybe what you might consider superficially
at first. Does it use the correct coding style? Does it re-invent
stuff which already exists in the kernel? Does its kAPI make sense,
does it abuse debugs or sysfs, etc. Also, do the unit tests fit in
nicely with other tests in tools/testing/selftests/net so they can be
run regularly. Until all that is dealt with, you might find that
nobody looks at the details of the protocols themselves.

> Most of these are already done; the ones that aren't (e.g., reverse
> Xmas-tree formatting, which I only recently discovered) will be done
> before I submit anything.

https://www.kernel.org/doc/html/latest/process/maintainer-netdev.html

ABIs tend to get the most scrutiny, since they are the most important
to get right, before they are fixed in stone. So try to make them easy
to find in the patchset.

	Andrew