netdev - Re: [PATCH bpf-next 1/4] xdp: Support specifying expected existing program when attaching XDP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3aca04e2-4034-f41a-8e98-f40471601dff@iogearbox.net>
Date:   Fri, 20 Mar 2020 22:55:43 +0100
From:   Daniel Borkmann <daniel@...earbox.net>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     Toke Høiland-Jørgensen <toke@...hat.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Andrii Nakryiko <andriin@...com>,
        "David S. Miller" <davem@...emloft.net>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        John Fastabend <john.fastabend@...il.com>,
        Lorenz Bauer <lmb@...udflare.com>,
        Andrey Ignatov <rdna@...com>, netdev@...r.kernel.org,
        bpf@...r.kernel.org
Subject: Re: [PATCH bpf-next 1/4] xdp: Support specifying expected existing
 program when attaching XDP

On 3/20/20 10:30 PM, Jakub Kicinski wrote:
> On Fri, 20 Mar 2020 21:40:46 +0100 Daniel Borkmann wrote:
>> On 3/20/20 9:30 PM, Daniel Borkmann wrote:
>>> On 3/20/20 9:48 AM, Toke Høiland-Jørgensen wrote:
>>>> Jakub Kicinski <kuba@...nel.org> writes:
>>>>> On Thu, 19 Mar 2020 14:13:13 +0100 Toke Høiland-Jørgensen wrote:
>>>>>> From: Toke Høiland-Jørgensen <toke@...hat.com>
>>>>>>
>>>>>> While it is currently possible for userspace to specify that an existing
>>>>>> XDP program should not be replaced when attaching to an interface, there is
>>>>>> no mechanism to safely replace a specific XDP program with another.
>>>>>>
>>>>>> This patch adds a new netlink attribute, IFLA_XDP_EXPECTED_FD, which can be
>>>>>> set along with IFLA_XDP_FD. If set, the kernel will check that the program
>>>>>> currently loaded on the interface matches the expected one, and fail the
>>>>>> operation if it does not. This corresponds to a 'cmpxchg' memory operation.
>>>>>>
>>>>>> A new companion flag, XDP_FLAGS_EXPECT_FD, is also added to explicitly
>>>>>> request checking of the EXPECTED_FD attribute. This is needed for userspace
>>>>>> to discover whether the kernel supports the new attribute.
>>>>>>
>>>>>> Signed-off-by: Toke Høiland-Jørgensen <toke@...hat.com>
>>>>>
>>>>> I didn't know we wanted to go ahead with this...
>>>>
>>>> Well, I'm aware of the bpf_link discussion, obviously. Not sure what's
>>>> happening with that, though. So since this is a straight-forward
>>>> extension of the existing API, that doesn't carry a high implementation
>>>> cost, I figured I'd just go ahead with this. Doesn't mean we can't have
>>>> something similar in bpf_link as well, of course.
>>>
>>> Overall series looks okay, but before we go down that road, especially given there is
>>> the new bpf_link object now, I would like us to first elaborate and figure out how XDP
>>> fits into the bpf_link concept, where its limitations are, whether it even fits at all,
>>> and how its semantics should look like realistically given bpf_link is to be generic to
>>> all program types. Then we could extend the atomic replace there generically as well. I
>>> think at the very minimum it might have similarities with what is proposed here, but
>>> from a user experience I would like to avoid having something similar in XDP API and
>>> then again in bpf_link which would just be confusing..
>>
>> Another aspect that falls into this atomic replacement is also that the programs can
>> actually be atomically replaced at runtime. Last time I looked, some drivers still do
>> a down/up cycle on replacement and hence traffic would be interrupted. I would argue
>> that such /atomic/ swap operation on bpf_link would cover a guarantee of not having to
>> perform this as well (workaround today would be a simple tail call map as entry point).
> 
> I don't think that's the case. Drivers generally have a fast path
> for the active-active replace.
> 
> Up/Down is only done to remap DMA buffers and change RX buffer
> allocation scheme. That's when program is installed or removed,
> not replaced.

I know; though it seems not all adhere to that scheme sadly. I don't have that HW so can
only judge on the code, but one example that looked suspicious enough to me is qede_xdp().
It calls qede_xdp_set(), which does a qede_reload() for /every/ prog update. The latter
basically does ...

     if (edev->state == QEDE_STATE_OPEN) {
         qede_unload(edev, QEDE_UNLOAD_NORMAL, true);
         if (args)
             args->func(edev, args);               <-- prog replace here
         qede_load(edev, QEDE_LOAD_RELOAD, true);
         [...]
     }

... now that is one driver. I haven't checked all the others (aside from i40e/ixgbe/mlx4/
mlx5/nfp), but in any case it's also fixable in the driver w/o the extra need for bpf_link.

Thanks,
Daniel