Date:   Wed, 01 Feb 2023 19:08:39 +0100
From:   Toke Høiland-Jørgensen <toke@...hat.com>
To:     Jamal Hadi Salim <jhs@...atatu.com>
Cc:     Jiri Pirko <jiri@...nulli.us>,
        John Fastabend <john.fastabend@...il.com>,
        Jamal Hadi Salim <hadi@...atatu.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Stanislav Fomichev <sdf@...gle.com>,
        Jakub Kicinski <kuba@...nel.org>, netdev@...r.kernel.org,
        kernel@...atatu.com, deb.chatterjee@...el.com,
        anjali.singhai@...el.com, namrata.limaye@...el.com,
        khalidm@...dia.com, tom@...anda.io, pratyush@...anda.io,
        xiyou.wangcong@...il.com, davem@...emloft.net, edumazet@...gle.com,
        pabeni@...hat.com, vladbu@...dia.com, simon.horman@...igine.com,
        stefanc@...vell.com, seong.kim@....com, mattyk@...dia.com,
        dan.daly@...el.com, john.andy.fingerhut@...el.com
Subject: Re: [PATCH net-next RFC 00/20] Introducing P4TC

Jamal Hadi Salim <jhs@...atatu.com> writes:

> On Tue, Jan 31, 2023 at 5:54 PM Toke Høiland-Jørgensen <toke@...hat.com> wrote:
>>
>> Jamal Hadi Salim <jhs@...atatu.com> writes:
>>
>> > So while going through this thought process, things to consider:
>> > 1) The autonomy of the tc infra, essentially the skip_sw/hw  controls
>> > and their packet driven iteration. Perhaps (the patch i pointed to
>> > from Paul Blakey) where part of the action graph runs in sw.
>>
>> Yeah, I agree that mixed-mode operation is an important consideration,
>> and presumably attaching metadata directly to a packet on the hardware
>> side, and accessing that in sw, is in scope as well? We seem to have
>> landed on exposing that sort of thing via kfuncs in XDP, so expanding on
>> that seems reasonable at a first glance.
>
> There is  built-in metadata chain id/prio/protocol (stored in cls
> common struct) passed when the policy is installed. The hardware may
> be able to handle received (probably packet encapsulated, but i
> believe that is vendor specific) metadata and transform it into the
> appropriate continuation point. Maybe a simpler example is to look at
> the patch from Paul (since that is the most recent change, so it is
> sticking in my brain); if you can follow the example,  you'll see
> there's some state that is transferred for the action with a cookie
> from/to the driver.

Right, that roughly fits my understanding. Just adding a kfunc to fetch
that cookie would be the obvious way to expose it to BPF.

>> > 2) The dynamicity of being able to trigger table offloads and/or
>> > kernel table updates which are packet driven (consider scenario where
>> > they have iterated the hardware and ingressed into the kernel).
>>
>> That could be done by either interface, though: the kernel can propagate
>> a bpf_map_update() from a BPF program to the hardware version of the
>> table as well. I suspect a map-based API at least on the BPF side would
>> be more natural, but I don't really have a strong opinion on this :)
>
> Should have mentioned this earlier as requirement:
> Speed of update is _extremely_ important, i.e how fast you can update
> could make or break things; see talk from Marcelo/Vlad[1]. My gut
> feeling is dealing with feedback from some vendor firmware/driver
> interface that the entry is really offloaded may cause challenges for
> ebpf by stalling the program. We have seen upto several ms delays on
> occasions.

Right, understandable. That seems kinda orthogonal to which API is used
to expose this data, though? In the end it's all just kernel code, and,
well, if updating things in an offloaded map/table is taking too long,
we'll have to either fix the underlying code to make it faster, or the
application will have keep things only in software? :)

-Toke

Powered by blists - more mailing lists