lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAM0EoMkV6STz+aGuZ8vtHHXSkS0yQJpcre32B5erLbqRiY6Azg@mail.gmail.com> Date: Thu, 16 Feb 2023 15:24:01 -0500 From: Jamal Hadi Salim <jhs@...atatu.com> To: Jamal Hadi Salim <hadi@...atatu.com> Cc: Edward Cree <ecree.xilinx@...il.com>, Toke Høiland-Jørgensen <toke@...hat.com>, Jiri Pirko <jiri@...nulli.us>, John Fastabend <john.fastabend@...il.com>, Willem de Bruijn <willemb@...gle.com>, Stanislav Fomichev <sdf@...gle.com>, Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org, kernel@...atatu.com, deb.chatterjee@...el.com, anjali.singhai@...el.com, namrata.limaye@...el.com, khalidm@...dia.com, tom@...anda.io, pratyush@...anda.io, xiyou.wangcong@...il.com, davem@...emloft.net, edumazet@...gle.com, pabeni@...hat.com, vladbu@...dia.com, simon.horman@...igine.com, stefanc@...vell.com, seong.kim@....com, mattyk@...dia.com, dan.daly@...el.com, john.andy.fingerhut@...el.com, "Jain, Vipin" <Vipin.Jain@....com> Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC Hi, Want to provide an update to this thread and a summary of where we are (typing this on web browser client so i hope it doesnt come all mangled up): I have had high bandwidth discussions with several people offlist (thanks to everyone who invested their time in trying to smoothen this); sometimes cooler headers prevail this way. We are willing (and are starting) to invest time to see how we can fit ebpf for the software datapath. Should be noted that we did look at ebpf when this project started and we ended up not going that path. I think what is new in this equation is the concept of kfuncs - which we didnt have back then. Perhaps with kfuncs we can make both worlds work together. XDP as well is appealing. As i have stated earlier: The starting premise is that the posted code meets our requirements so whatever we do using ebpf has to meet our requirements. I am ok with some limited degree of square hole round peg situation but it cant be interfering in meeting our goals. So let me restate those goals so we dont go into some rabit hole in the discussion: 1) Supporting P4 in the kernel both for the sw and hw datapath utilizing the well established tc infra which allows both sw equivalence and hw offload. We are _not_ going to reinvent this. Essentially we get the whole package: from the control plane to the tooling infra, netlink messaging to s/w and h/w symbiosis, the autonomous kernel control, etc. The advantage is that we have a singular vendor-neutral interface via the kernel using well understood mechanisms. Behavioral equivalence between hw and sw is a given. 2) Operational usability - this is encoded currently in the scriptability approach. Ex, I can just ship someone a shell script in an email but more important if they have deployed tc offloads the runtime semantics are unchanged. "write once, run anywhere" paradigm is easier to state in ascii;-> The interface is designed to be scriptable to remove the burden of making kernel and user space code changes for any new processing functions (whether in s/w or hardware). 3) Debuggability - developers and ops people who are familiar with tc offloads can continue using the _same existing techniques and tools_. This also eases support. 4) Performance - note our angle on this, based on the niche we are looking at is "if you want performance then offload". However, one discussion point that has been raised multiple times in the thread and in private is that there are performance gains when using ebpf. This arguement is reasonable and a motivator for us to invest our time in evaluating. We have started doing off the cuff measurements. With very simple P4 program which receives a packet, looks up a table, and on a hit changes src/mac address then forwards. We have: A) implemented a handcoded ebpf program, B) generated P4TC sw only C) flower s/w only (skip_hw) rules and D) hardware offload (skip_sw) all on tc (so we can do orange-orange comparison). The SUT has a dual port CX6 NIC capable of offloading pedit and mirred. Trex is connected to one port and sending http gets which goes via the box and a response comes back on the other port which we send back to trex. The traffic is very asymettric; data coming back to the client fills up the 25G pipe but ACKs going back consume a lot less. Unfortunately all 4 scenarios were able to handle the wire rate - we are going to set up more nasty traffic generation later; for now we opted to look at cpu utilization for the 4 scenarios. We have the following results: A) 35% CPU B) 39% C) 36% D) 0% This is by no means a good test but i wanted to illustrate the relevance of #D (0%) - which is a main itch for us. We need to test more complex programs which is where probably the performance of ebpf will shine. XDP for sure will beat all the others - but i would rather get the facts in place first. So we are investing effort in this direction and will share results at some point. There may be other low hanging fruits that have been brought up in the discussion for ebpf (the parser being one); we will be looking at all those as well. Note: The goal of this exercise for us is to evaluate not just performance but also consider how it affects the other P4TC goals. There may be a sweet happy point somewhere in there but we need to collect the data instead of hypothesizing. cheers, jamal
Powered by blists - more mailing lists