linux-kernel - Re: [PATCH bpf-next v3] bpf/docs: Document kfunc lifecycle / stability expectations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87mt5rdd8i.fsf@toke.dk>
Date:   Sun, 05 Feb 2023 22:16:45 +0100
From:   Toke Høiland-Jørgensen <toke@...hat.com>
To:     Cong Wang <xiyou.wangcong@...il.com>,
        David Vernet <void@...ifault.com>
Cc:     bpf@...r.kernel.org, ast@...nel.org, daniel@...earbox.net,
        andrii@...nel.org, martin.lau@...ux.dev, song@...nel.org,
        yhs@...a.com, john.fastabend@...il.com, kpsingh@...nel.org,
        sdf@...gle.com, haoluo@...gle.com, jolsa@...nel.org,
        linux-kernel@...r.kernel.org, kernel-team@...a.com, corbet@....net,
        linux-doc@...r.kernel.org, brouer@...hat.com, bagasdotme@...il.com,
        linux-api@...r.kernel.org
Subject: Re: [PATCH bpf-next v3] bpf/docs: Document kfunc lifecycle /
 stability expectations

Cong Wang <xiyou.wangcong@...il.com> writes:

> On Fri, Feb 03, 2023 at 09:57:27AM -0600, David Vernet wrote:
>> BPF kernel <-> kernel API stability has been discussed at length over
>> the last several weeks and months. Now that we've largely aligned over
>> kfuncs being the way forward, and BPF helpers being considered
>> functionally frozen, it's time to document the expectations for kfunc
>> lifecycles and stability so that everyone (BPF users, kfunc developers,
>> and maintainers) are all aligned, and have a crystal-clear understanding
>> of the expectations surrounding kfuncs.
>> 
>> To do that, this patch adds that documentation to the main kfuncs
>> documentation page via a new 'kfunc lifecycle expectations' section. The
>> patch describes how decisions are made in the kernel regarding whether
>> to include, keep, deprecate, or change / remove a kfunc. As described
>> very overtly in the patch itself, but likely worth highlighting here:
>> 
>> "kfunc stability" does not mean, nor ever will mean, "BPF APIs may block
>> development elsewhere in the kernel".
>> 
>> Rather, the intention and expectation is for kfuncs to be treated like
>> EXPORT_SYMBOL_GPL symbols in the kernel. The goal is for kfuncs to be a
>> safe and valuable option for maintainers and kfunc developers to extend
>> the kernel, without tying anyone's hands, or imposing any kind of
>> restrictions on maintainers in the same way that UAPI changes do.
>
> I think they are still different, kernel modules are still considered as
> a part of kernel development, while eBPF code is not that supposed to be
> kernel development, at least much further. Treating them alike is
> misleading, IMHO.

If you read the actual documentation text added to kfuncs.rst this
difference is indeed called out. But you're right that "treated like" in
the commit message is probably a bit strong.

>> In addition to the 'kfunc lifecycle expectations' section, this patch
>> also adds documentation for a new KF_DEPRECATED kfunc flag which kfunc
>> authors or maintainers can choose to add to kfuncs if and when they
>> decide to deprecate them. Note that as described in the patch itself, a
>> kfunc need not be deprecated before being changed or removed -- this
>> flag is simply provided as an available deprecation mechanism for those
>> that want to provide a deprecation story / timeline to their users.
>> When necessary, kfuncs may be changed or removed to accommodate changes
>> elsewhere in the kernel without any deprecation at all.
>
> This fundamentally contradicts with Compile-Once-Run-Everywhere
> https://facebookmicrosites.github.io/bpf/blog/2020/02/19/bpf-portability-and-co-re.html
> Could you add some clarification for this too? Especically how we could
> respect CO-RE meanwhile deprecating kfuncs?

Well, CO-RE doesn't work for kfuncs, currently, so... :)

What do you mean "respect CO-RE", though? CO-RE is a tool to make BPF
programs more portable, so not sure how one would "respect" that?

> BTW, not related to compatibility, but still kfuncs related confusion,
> it also contradicts with Documentation/bpf/bpf_design_QA.rst:
>
> "
> Q: Can BPF functionality such as new program or map types, new
> helpers, etc be added out of kernel module code?
>
> A: NO.
> "
>
> The conntrack kfuncs like bpf_skb_ct_alloc() reside in a kernel
> module.

Yup, good point, we should update that. I'll send a patch...

-Toke