Date:   Thu, 2 Feb 2023 15:34:26 -0800
From:   Tom Herbert <tom@...anda.io>
To:     Jamal Hadi Salim <jhs@...atatu.com>
Cc:     Toke Høiland-Jørgensen <toke@...hat.com>,
        Jiri Pirko <jiri@...nulli.us>,
        John Fastabend <john.fastabend@...il.com>,
        Jamal Hadi Salim <hadi@...atatu.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Stanislav Fomichev <sdf@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org,
        kernel@...atatu.com, deb.chatterjee@...el.com,
        anjali.singhai@...el.com, namrata.limaye@...el.com,
        khalidm@...dia.com, pratyush@...anda.io, xiyou.wangcong@...il.com,
        davem@...emloft.net, edumazet@...gle.com, pabeni@...hat.com,
        vladbu@...dia.com, simon.horman@...igine.com, stefanc@...vell.com,
        seong.kim@....com, mattyk@...dia.com, dan.daly@...el.com,
        john.andy.fingerhut@...el.com
Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC

On Thu, Feb 2, 2023 at 10:51 AM Jamal Hadi Salim <jhs@...atatu.com> wrote:
>
> Sorry I was distracted somewhere else.
> I am not sure i fully grokked your proposal but I am willing to go
> through this thought exercise with you (perhaps a higher bandwidth
> media would help); however,  we should put some parameters so it
> doesnt become a perpetual discussion:
>
> The starting premise is that posted code meets our requirements so
> whatever we do using ebpf has to meet our requirements; we dont want
> to get into a wrestling match with any of the ebpf constraints.
> Actually, I am ok with some limited degree of square hole round peg
> situation but it cant be interfering in getting work done. I would
> also be ok with small surgeries into the ebpf core if needed to meet
> our requirements.

Can you elaborate on what the problems are with using eBPF? I know
there is at least one P4->eBPF compiler, what is lacking that doesn't
meet your requirements?

> Performance and maintainability are also on the table.

Performance of the software datapath is of paramount importance. My
fundamental concern here is that if we push an underperforming
software solution, then the patches don't just enable offload, they'll
be used to *justify* it. That is, the hardware vendors might go to
their customers and show how much better the offload is than the slow
software solution; whereas if they compared to a higher performing
software solution it might meet the performance requirements of the
customer thereby saving them the cost and complexity of offload. Note
we've already been down this path once with DPDK once being touted as
being "10x faster than the kernel" with little regard to whether the
kernel could be tuned or adapted-- of course, we subsequently invented
XDP and pretty much closed the gap.

Tom

>
> Let me know what you think.
>
> cheers,
> jamal
>
> On Wed, Feb 1, 2023 at 1:08 PM Toke Høiland-Jørgensen <toke@...hat.com> wrote:
> >
> > Jamal Hadi Salim <jhs@...atatu.com> writes:
> >
> > > On Tue, Jan 31, 2023 at 5:54 PM Toke Høiland-Jørgensen <toke@...hat.com> wrote:
> > >>
> > >> Jamal Hadi Salim <jhs@...atatu.com> writes:
> > >>
> > >> > So while going through this thought process, things to consider:
> > >> > 1) The autonomy of the tc infra, essentially the skip_sw/hw  controls
> > >> > and their packet driven iteration. Perhaps (the patch i pointed to
> > >> > from Paul Blakey) where part of the action graph runs in sw.
> > >>
> > >> Yeah, I agree that mixed-mode operation is an important consideration,
> > >> and presumably attaching metadata directly to a packet on the hardware
> > >> side, and accessing that in sw, is in scope as well? We seem to have
> > >> landed on exposing that sort of thing via kfuncs in XDP, so expanding on
> > >> that seems reasonable at a first glance.
> > >
> > > There is  built-in metadata chain id/prio/protocol (stored in cls
> > > common struct) passed when the policy is installed. The hardware may
> > > be able to handle received (probably packet encapsulated, but i
> > > believe that is vendor specific) metadata and transform it into the
> > > appropriate continuation point. Maybe a simpler example is to look at
> > > the patch from Paul (since that is the most recent change, so it is
> > > sticking in my brain); if you can follow the example,  you'll see
> > > there's some state that is transferred for the action with a cookie
> > > from/to the driver.
> >
> > Right, that roughly fits my understanding. Just adding a kfunc to fetch
> > that cookie would be the obvious way to expose it to BPF.
> >
> > >> > 2) The dynamicity of being able to trigger table offloads and/or
> > >> > kernel table updates which are packet driven (consider scenario where
> > >> > they have iterated the hardware and ingressed into the kernel).
> > >>
> > >> That could be done by either interface, though: the kernel can propagate
> > >> a bpf_map_update() from a BPF program to the hardware version of the
> > >> table as well. I suspect a map-based API at least on the BPF side would
> > >> be more natural, but I don't really have a strong opinion on this :)
> > >
> > > Should have mentioned this earlier as requirement:
> > > Speed of update is _extremely_ important, i.e how fast you can update
> > > could make or break things; see talk from Marcelo/Vlad[1]. My gut
> > > feeling is dealing with feedback from some vendor firmware/driver
> > > interface that the entry is really offloaded may cause challenges for
> > > ebpf by stalling the program. We have seen upto several ms delays on
> > > occasions.
> >
> > Right, understandable. That seems kinda orthogonal to which API is used
> > to expose this data, though? In the end it's all just kernel code, and,
> > well, if updating things in an offloaded map/table is taking too long,
> > we'll have to either fix the underlying code to make it faster, or the
> > application will have keep things only in software? :)
> >
> > -Toke
> >

Powered by blists - more mailing lists