netdev - Re: [PATCH bpf v4 1/3] bpf, cgroups: Fix cgroup v2 fallback on v1/v2 mixed mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAADnVQKHcpmUp8eDzmJ2ktdtYg8ChnqK44W0_MrAQ95Y9VXVqQ@mail.gmail.com>
Date:   Mon, 13 Sep 2021 19:01:24 -0700
From:   Alexei Starovoitov <alexei.starovoitov@...il.com>
To:     Daniel Borkmann <daniel@...earbox.net>
Cc:     bpf <bpf@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Tejun Heo <tj@...nel.org>,
        "David S. Miller" <davem@...emloft.net>,
        Martynas Pumputis <m@...bda.lt>,
        Andrii Nakryiko <andrii@...nel.org>,
        Stanislav Fomichev <sdf@...gle.com>
Subject: Re: [PATCH bpf v4 1/3] bpf, cgroups: Fix cgroup v2 fallback on v1/v2
 mixed mode

On Mon, Sep 13, 2021 at 4:08 PM Daniel Borkmann <daniel@...earbox.net> wrote:
>
> Fix cgroup v1 interference when non-root cgroup v2 BPF programs are used.
> Back in the days, commit bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
> embedded per-socket cgroup information into sock->sk_cgrp_data and in order
> to save 8 bytes in struct sock made both mutually exclusive, that is, when
> cgroup v1 socket tagging (e.g. net_cls/net_prio) is used, then cgroup v2
> falls back to the root cgroup in sock_cgroup_ptr() (&cgrp_dfl_root.cgrp).
>
> The assumption made was "there is no reason to mix the two and this is in line
> with how legacy and v2 compatibility is handled" as stated in bd1060a1d671.
> However, with Kubernetes more widely supporting cgroups v2 as well nowadays,
> this assumption no longer holds, and the possibility of the v1/v2 mixed mode
> with the v2 root fallback being hit becomes a real security issue.
>
> Many of the cgroup v2 BPF programs are also used for policy enforcement, just
> to pick _one_ example, that is, to programmatically deny socket related system
> calls like connect(2) or bind(2). A v2 root fallback would implicitly cause
> a policy bypass for the affected Pods.
>
> In production environments, we have recently seen this case due to various
> circumstances: i) a different 3rd party agent and/or ii) a container runtime
> such as [0] in the user's environment configuring legacy cgroup v1 net_cls
> tags, which triggered implicitly mentioned root fallback. Another case is
> Kubernetes projects like kind [1] which create Kubernetes nodes in a container
> and also add cgroup namespaces to the mix, meaning programs which are attached
> to the cgroup v2 root of the cgroup namespace get attached to a non-root
> cgroup v2 path from init namespace point of view. And the latter's root is
> out of reach for agents on a kind Kubernetes node to configure. Meaning, any
> entity on the node setting cgroup v1 net_cls tag will trigger the bypass
> despite cgroup v2 BPF programs attached to the namespace root.
>
> Generally, this mutual exclusiveness does not hold anymore in today's user
> environments and makes cgroup v2 usage from BPF side fragile and unreliable.
> This fix adds proper struct cgroup pointer for the cgroup v2 case to struct
> sock_cgroup_data in order to address these issues; this implicitly also fixes
> the tradeoffs being made back then with regards to races and refcount leaks
> as stated in bd1060a1d671, and removes the fallback, so that cgroup v2 BPF
> programs always operate as expected.
>
>   [0] https://github.com/nestybox/sysbox/
>   [1] https://kind.sigs.k8s.io/
>
> Fixes: bd1060a1d671 ("sock, cgroup: add sock->sk_cgroup")
> Signed-off-by: Daniel Borkmann <daniel@...earbox.net>
> Acked-by: Stanislav Fomichev <sdf@...gle.com>
> Acked-by: Tejun Heo <tj@...nel.org>
> Cc: David S. Miller <davem@...emloft.net>
> Cc: Martynas Pumputis <m@...bda.lt>

Applied. Thanks!