Message-ID: <CALx6S36akGgRKohduW-aApgRCbZqjJ5uDzTeGQpD=pPietN2Dg@mail.gmail.com>
Date:   Sat, 28 Jan 2023 13:17:35 -0800
From:   Tom Herbert <tom@...bertland.com>
To:     "Singhai, Anjali" <anjali.singhai@...el.com>
Cc:     Jamal Hadi Salim <hadi@...atatu.com>,
        Jakub Kicinski <kuba@...nel.org>,
        Jamal Hadi Salim <jhs@...atatu.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "kernel@...atatu.com" <kernel@...atatu.com>,
        "Chatterjee, Deb" <deb.chatterjee@...el.com>,
        "Limaye, Namrata" <namrata.limaye@...el.com>,
        "khalidm@...dia.com" <khalidm@...dia.com>,
        "tom@...anda.io" <tom@...anda.io>,
        "pratyush@...anda.io" <pratyush@...anda.io>,
        "jiri@...nulli.us" <jiri@...nulli.us>,
        "xiyou.wangcong@...il.com" <xiyou.wangcong@...il.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "edumazet@...gle.com" <edumazet@...gle.com>,
        "pabeni@...hat.com" <pabeni@...hat.com>,
        "vladbu@...dia.com" <vladbu@...dia.com>,
        "simon.horman@...igine.com" <simon.horman@...igine.com>,
        "stefanc@...vell.com" <stefanc@...vell.com>,
        "seong.kim@....com" <seong.kim@....com>,
        "mattyk@...dia.com" <mattyk@...dia.com>,
        "Daly, Dan" <dan.daly@...el.com>,
        "Fingerhut, John Andy" <john.andy.fingerhut@...el.com>
Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC

On Fri, Jan 27, 2023 at 5:34 PM Singhai, Anjali
<anjali.singhai@...el.com> wrote:
>
> P4 is definitely the language of choice for defining a Dataplane in HW for IPUs/DPUs/FNICs and Switches. As a vendor I can definitely say that the smart devices implement a very programmable ASIC as each customer Dataplane defers quite a bit and P4 is the language of choice for specifying the Dataplane definitions. A lot of customer deploy proprietary protocols that run in HW and there is no good way right now in kernel to support these proprietary protcols. If we enable these protocol in the kernel it takes a huge effort and they don’t evolve well.
> Being able to define in P4 and offload into HW using tc mechanism really helps in supporting the customer's Dataplane and protcols without having to wait months and years to get the kernel updated. Here is a link to our IPU offering that is P4 programmable

Anjali,

P4 may be the language of choice for programming HW datapath, however
it's not the language of choice for programming SW datapaths-- that's
C over XDP/eBPF. And while XDP/eBPF also doesn't depend on kernel
updates, it has a major advantage over P4 in that it doesn't require
fancy hardware either.

Even at full data center deployment of P4 devices, there will be at
least an order of magnitude more deployment of SW programmed
datapaths; and unless someone is using P4 hardware, there's zero value
in rewriting programs in P4 instead of C. IMO, we will never see
networking developers moving to P4 en masse-- P4 will always be a
niche market relative to the programmable datapath space and the skill
sets required to support serious scalable deployment. That being said,
there will be a nontrivial contingent of users who need to run the
same programs in both SW and HW environments. Expecting them to
maintain two very different code bases to support two disparate models
is costly and prohibitive to them. So for their benefit, we need a
solution to reconcile these two models. P4TC is one means to
accomplish that.

We want to consider both the permutations: 1) compile C code to run in
P4 hardware 2) compile P4 to run in SW. If we establish a common IR,
then we can generalize the problem: programmer writes their datapath
in the language of their choosing (P4, C, Python, Rust, etc.), they
compile the program to whatever backend they are using (HW, SW,
XDP/eBPF, etc.). The P4TC CLI serves as one such IR as there's nothing
that prevents someone from compiling a program from another language
to the CLI (for instance, we've implemented the compiler to output the
parser CLI from PANDA-C). The CLI natively runs in kernel SW, and with
the right hooks could be offloaded to HW-- not just P4 hardware but
potentially other hardware targets as well.

Tom

>  https://www.intel.com/content/www/us/en/products/details/network-io/ipu/e2000-asic.html
> Here are some other useful links
> https://ipdk.io/
>
> Anjali
>
> -----Original Message-----
> From: Jamal Hadi Salim <hadi@...atatu.com>
> Sent: Friday, January 27, 2023 11:43 AM
> To: Jakub Kicinski <kuba@...nel.org>
> Cc: Jamal Hadi Salim <jhs@...atatu.com>; netdev@...r.kernel.org; kernel@...atatu.com; Chatterjee, Deb <deb.chatterjee@...el.com>; Singhai, Anjali <anjali.singhai@...el.com>; Limaye, Namrata <namrata.limaye@...el.com>; khalidm@...dia.com; tom@...anda.io; pratyush@...anda.io; jiri@...nulli.us; xiyou.wangcong@...il.com; davem@...emloft.net; edumazet@...gle.com; pabeni@...hat.com; vladbu@...dia.com; simon.horman@...igine.com; stefanc@...vell.com; seong.kim@....com; mattyk@...dia.com; Daly, Dan <dan.daly@...el.com>; Fingerhut, John Andy <john.andy.fingerhut@...el.com>
> Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC
>
> On Fri, Jan 27, 2023 at 12:18 PM Jakub Kicinski <kuba@...nel.org> wrote:
> >
> > On Fri, 27 Jan 2023 08:33:39 -0500 Jamal Hadi Salim wrote:
> > > On Thu, Jan 26, 2023 at 6:30 PM Jakub Kicinski <kuba@...nel.org> wrote:
> > > > On Tue, 24 Jan 2023 12:03:46 -0500 Jamal Hadi Salim wrote:
>
> [..]
> > > Network programmability involving hardware  - where at minimal the
> > > specification of the datapath is in P4 and often the implementation
> > > is. For samples of specification using P4 (that are public) see for
> > > example MS Azure:
> > > https://github.com/sonic-net/DASH/tree/main/dash-pipeline
> >
> > That's an IPU thing?
> >
>
> Yes, DASH is xPU. But the whole Sonic/SAI thing includes switches and P4 plays a role there.
>
> > > If you are a vendor and want to sell a NIC in that space, the spec
> > > you get is in P4.
> >
> > s/NIC/IPU/ ?
>
> I do believe that one can write a P4 program to express things a regular NIC could express that may be harder to expose with current interfaces.
>
> > > Your underlying hardware
> > > doesnt have to be P4 native, but at minimal the abstraction (as we
> > > are trying to provide with P4TC) has to be able to consume the P4
> > > specification.
> >
> > P4 is certainly an option, especially for specs, but I haven't seen
> > much adoption myself.
>
> The xPU market outside of hyper-scalers is emerging now. Hyperscalers looking at xPUs are looking at P4 as the datapath language - that sets the trend forward to large enterprises.
> That's my experience.
> Some of the vendors on the Cc should be able to point to adoption.
> Anjali? Matty?
>
> > What's the benefit / use case?
>
> Of P4 or xPUs?
> Unified approach to standardize how a datapath is defined is a value for P4.
> Providing a singular abstraction via the kernel (as opposed to every vendor pitching their API) is what the kernel brings.
>
> > > For implementations where P4 is in use, there are many - some public
> > > others not, sample space:
> > > https://cloud.google.com/blog/products/gcp/google-cloud-using-p4runt
> > > ime-to-build-smart-networks
> >
> > Hyper-scaler proprietary.
>
> The control abstraction (P4 runtime) is certainly not proprietary.
> The datapath that is targetted by the runtime is.
> Hopefully we can fix that with P4TC.
> The majority of the discussions i have with some of the folks who do kernel bypass have one theme in common:
> The kernel process is just too long. Trying to add one feature to flower could take anywhere from 6 months to 3 years to finally show up in some supported distro. With P4TC we are taking the approach of scriptability to allow for speacilized datapaths (which P4 excels in). The google datapath maybe proprietary while their hardware may even(or not) be using native P4 - but the important detail is we have _a way_ to abstract those datapaths.
>
> > > There are NICs and switches which are P4 native in the market.
> >
> > Link to docs?
> >
>
> Off top of my head Intel Mount Evans, Pensando, Xilinx FPGAs, etc. The point is to bring them together under the linux umbrella.
>
> > > IOW, there is beacoup $ investment in this space that makes it worth pursuing.
> >
> > Pursuing $ is good! But the community IMO should maximize a different
> > function.
>
> While I agree $ is not the primary motivator it is a factor, it is a good indicator. No different than the network stack being tweaked to do certain things that certain hyperscalers need because they invest $.
> I have no problems with a large harmonious tent.
>
> cheers,
> jamal
>
> > > TC is the kernel offload mechanism that has gathered deployment
> > > experience over many years - hence P4TC.
> >
> > I don't wanna argue. I thought it'd be more fair towards you if I made
> > my lack of conviction known, rather than sit quiet and ignore it since
> > it's just an RFC.

Powered by blists - more mailing lists