Date: Tue, 04 Jun 2024 13:32:21 +0200
From: Niklas Schnelle <schnelle@...ux.ibm.com>
To: "D. Wythe" <alibuda@...ux.alibaba.com>, kgraul@...ux.ibm.com,
        wenjia@...ux.ibm.com, jaka@...ux.ibm.com, wintera@...ux.ibm.com,
        guwen@...ux.alibaba.com
Cc: kuba@...nel.org, davem@...emloft.net, netdev@...r.kernel.org,
        linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org,
        tonylu@...ux.alibaba.com, pabeni@...hat.com, edumazet@...gle.com
Subject: Re: [PATCH net-next v5 0/3] Introduce IPPROTO_SMC

On Mon, 2024-06-03 at 23:07 +0800, D. Wythe wrote:
> 
> On 6/3/24 3:48 PM, Niklas Schnelle wrote:
> > On Thu, 2024-05-30 at 18:14 +0800, D. Wythe wrote:
> > > On 5/30/24 5:30 PM, D. Wythe wrote:
> > > > From: "D. Wythe" <alibuda@...ux.alibaba.com>
> > > 
---8<---
> > Hi D. Wythe,
> > 
> > I gave this series plus your smc_run.bpf and SMC_LO based SMC-D a test
> > run on my Ryzen 3900X workstation and I have to say I'm quite
> > impressed. I first tried the SMC_LO feature as merged in v6.10-rc1 with
> > the classic LD_PRELOAD based smc_run and iperf3, and qperf …
> > tcp_bw/tcp_lat both with normal localhost and between docker
> > containers. For this to work I of course had to initially set my UEID
> > as x86_64 unlike s390x doesn't get an SEID set. I used the following
> > script for this.
> > 
> > 
> > #!/usr/bin/sh
> > machine_id_upper=$(cat /etc/machine-id | tr '[:lower:]' '[:upper:]')
> > machine_id_suffix=$(echo "$machine_id_upper" | head -c 27)
> > ueid="MID-$machine_id_suffix"
> > smcd ueid add "$ueid"
> > 
> > 
> > The performance is pretty impressive:
> > * iperf3 with 12 parallel connections (matching core count) results in
> >    ~152 Gbit/s on normal loopback and ~312 Gbit/s with SMC_LO.
> > * qperf … tcp_bw (single thread) results in ~46 Gbit/s on normal loopback
> >    and ~58 Gbit/s with SMC_LO
> > * qperf … tcp_lat latency test results in 5-9 us with normal loopback
> >    and around 3-4 us with SMC_LO
> > 
> > Then I applied this series on top of v6.10-rc1 and tried it with your
> > smc_run.bpf. The performance is of course in-line with the above but
> > thanks to being able to enable SMC on a per-netns basis I was able to
> > try a few more thing. First I tried just enabling it in my default
> > netns and verified that after restarting sshd new ssh connections to
> > localhost used SMC-D through SMC_LO. Then I started Chrome and
> > confirmed that its TCP connections also registered with SMC and
> > successfully fell back to TCP mode. I had no trouble with normal
> > browsing though I guess especially Google stuff often uses HTTP/3 so
> > isn't affected. Still nice to see I didn't get breakage.
> > 
> > Secondly I tried smc_run.bpf with docker containers using the following
> > trick:
> > 
> > docker inspect --format '{{.State.Pid}}' <my_container_name>
> > 34651
> > nsenter -t 34651 -n smc_run.bpf -n 1
> > 
> > Sadly this only works for commands started in the container after
> > loading the BPF. So I wonder if you know of a good way to either
> > automatically execute smc_run.bpf on container start or maybe use it on
> > the docker daemon such that all namespaces created by docker get the
> > IPPROTO_SMC override. I'd then definitely consider using SMC-D with
> > SMC_LO between my home lab containers even if just for bragging rights
> > ;-)
> > 
> > Feel free to add for the IPPROTO_SMC series:
> > 
> > Tested-by: Niklas Schnelle <schnelle@...ux.ibm.com>
> > 
> > Thanks,
> > Niklas
> 
> Hi Niklas ,
> 
> Thanks very much for your testing.
> 
> Regarding your question, have you ever tried starting the container 
> using 'smc_run.bpf docker' ?
> 
> The smc_run.bpf allows the capability for replacement to be inherited by 
> descendant processes. This might meet your needs.
> However, it should be noted that this scope would no longer be limited 
> to netns.
> 
> If you don't want to replace the docker command and would like to keep 
> per netns, there are indeed some tricky ways, for example,
> we could check current process name when creating new netns to decide if 
> we should add it to the ebpf-map,
> but I think it's not appropriate to include this in smc_run.bpf.
> 
> Best wishes,
> D. Wythe

I'll have to try this. I'm guessing that for docker the smc_run would
have to be added for the docker daemon and not the individual docker
commands. For podman on the other hand the individual command might
work as there is no central daemon. And as you said bpf should allow us
to add other policies in the future.

Thanks,
Niklas

Powered by blists - more mailing lists