| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CALOAHbDwNRAn3kZ_H51NFbTXL4rZpVLNaCJoM8J0nB17wgrfTg@mail.gmail.com>
Date: Wed, 10 Aug 2022 23:21:33 +0800
From: Yafang Shao <laoar.shao@...il.com>
To: Alexei Starovoitov <ast@...nel.org>,
Daniel Borkmann <daniel@...earbox.net>,
Andrii Nakryiko <andrii@...nel.org>, Martin Lau <kafai@...com>,
Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
john fastabend <john.fastabend@...il.com>,
KP Singh <kpsingh@...nel.org>,
Stanislav Fomichev <sdf@...gle.com>,
Hao Luo <haoluo@...gle.com>, jolsa@...nel.org,
Johannes Weiner <hannes@...xchg.org>,
Michal Hocko <mhocko@...nel.org>,
Roman Gushchin <roman.gushchin@...ux.dev>,
Shakeel Butt <shakeelb@...gle.com>,
Muchun Song <songmuchun@...edance.com>,
Andrew Morton <akpm@...ux-foundation.org>
Cc: netdev <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
Linux MM <linux-mm@...ck.org>
Subject: Re: [PATCH 00/15] bpf: Introduce selectable memcg for bpf map
On Wed, Aug 10, 2022 at 11:13 PM Yafang Shao <laoar.shao@...il.com> wrote:
>
> On our production environment, we may load, run and pin bpf programs and
> maps in containers. For example, some of our networking bpf programs and
> maps are loaded and pinned by a process running in a container on our
> k8s environment. In this container, there're also running some other
> user applications which watch the networking configurations from remote
> servers and update them on this local host, log the error events, monitor
> the traffic, and do some other stuffs. Sometimes we may need to update
> these user applications to a new release, and in this update process we
> will destroy the old container and then start a new genration. In order not
> to interrupt the bpf programs in the update process, we will pin the bpf
> programs and maps in bpffs. That is the background and use case on our
> production environment.
>
> After switching to memcg-based bpf memory accounting to limit the bpf
> memory, some unexpected issues jumped out at us.
> 1. The memory usage is not consistent between the first generation and
> new generations.
> 2. After the first generation is destroyed, the bpf memory can't be
> limited if the bpf maps are not preallocated, because they will be
> reparented.
>
> This patchset tries to resolve these issues by introducing an
> independent memcg to limit the bpf memory.
>
> In the bpf map creation, we can assign a specific memcg instead of using
> the current memcg. That makes it flexible in containized environment.
> For example, if we want to limit the pinned bpf maps, we can use below
> hierarchy,
>
> Shared resources Private resources
>
> bpf-memcg k8s-memcg
> / \ /
> bpf-bar-memcg bpf-foo-memcg srv-foo-memcg
> | / \
> (charged) (not charged) (charged)
> | / \
> | / \
> bpf-foo-{progs, maps} srv-foo
>
> srv-foo loads and pins bpf-foo-{progs, maps}, but they are charged to an
> independent memcg (bpf-foo-memcg) instead of srv-foo's memcg
> (srv-foo-memcg).
>
> Pls. note that there may be no process in bpf-foo-memcg, that means it
> can be rmdir-ed by root user currently. Meanwhile we don't forcefully
> destroy a memcg if it doesn't have any residents. So this hierarchy is
> acceptible.
>
> In order to make the memcg of bpf maps seletectable, this patchset
> introduces some memory allocation wrappers to allocate map related
> memory. In these wrappers, it will get the memcg from the map and then
> charge the allocated pages or objs.
>
> Currenly it only supports for bpf map, and we can extend it to bpf prog
> as well. It only supports for cgroup2 now, but we can make an additional
> change in cgroup_get_from_fd() to support it for cgroup1.
>
> The observebility can also be supported in the next step, for example,
> showing the bpf map's memcg by 'bpftool map show' or even showing which
> maps are charged to a specific memcg by 'bpftool cgroup show'.
> Furthermore, we may also show an accurate memory size of a bpf map
> instead of an estimated memory size in 'bpftool map show' in the future.
>
> RFC->v1:
> - get rid of bpf_map container wrapper (Alexei)
> - add the new field into the end of struct (Alexei)
> - get rid of BPF_F_SELECTABLE_MEMCG (Alexei)
> - save memcg in bpf_map_init_from_attr
> - introduce bpf_ringbuf_pages_{alloc,free} and keep them inside
> kernel/bpf/ringbuf.c (Andrii)
>
> Yafang Shao (15):
> bpf: Remove unneeded memset in queue_stack_map creation
> bpf: Use bpf_map_area_free instread of kvfree
> bpf: Make __GFP_NOWARN consistent in bpf map creation
> bpf: Use bpf_map_area_alloc consistently on bpf map creation
> bpf: Fix incorrect mem_cgroup_put
> bpf: Define bpf_map_{get,put}_memcg for !CONFIG_MEMCG_KMEM
> bpf: Call bpf_map_init_from_attr() immediately after map creation
> bpf: Save memcg in bpf_map_init_from_attr()
> bpf: Use scoped-based charge in bpf_map_area_alloc
> bpf: Introduce new helpers bpf_ringbuf_pages_{alloc,free}
> bpf: Use bpf_map_kzalloc in arraymap
> bpf: Use bpf_map_kvcalloc in bpf_local_storage
> mm, memcg: Add new helper get_obj_cgroup_from_cgroup
> bpf: Add return value for bpf_map_init_from_attr
> bpf: Introduce selectable memcg for bpf map
>
> include/linux/bpf.h | 43 ++++++++++++-
> include/linux/memcontrol.h | 11 ++++
> include/uapi/linux/bpf.h | 1 +
> kernel/bpf/arraymap.c | 34 ++++++-----
> kernel/bpf/bloom_filter.c | 11 +++-
> kernel/bpf/bpf_local_storage.c | 17 ++++--
> kernel/bpf/bpf_struct_ops.c | 19 +++---
> kernel/bpf/cpumap.c | 17 ++++--
> kernel/bpf/devmap.c | 30 ++++++----
> kernel/bpf/hashtab.c | 26 ++++----
> kernel/bpf/local_storage.c | 12 ++--
> kernel/bpf/lpm_trie.c | 12 +++-
> kernel/bpf/offload.c | 12 ++--
> kernel/bpf/queue_stack_maps.c | 13 ++--
> kernel/bpf/reuseport_array.c | 11 +++-
> kernel/bpf/ringbuf.c | 104 ++++++++++++++++++++++----------
> kernel/bpf/stackmap.c | 13 ++--
> kernel/bpf/syscall.c | 133 ++++++++++++++++++++++++++++-------------
> mm/memcontrol.c | 41 +++++++++++++
> net/core/sock_map.c | 30 ++++++----
> net/xdp/xskmap.c | 12 +++-
> tools/include/uapi/linux/bpf.h | 1 +
> tools/lib/bpf/bpf.c | 3 +-
> tools/lib/bpf/bpf.h | 3 +-
> tools/lib/bpf/gen_loader.c | 2 +-
> tools/lib/bpf/libbpf.c | 2 +
> tools/lib/bpf/skel_internal.h | 2 +-
> 27 files changed, 436 insertions(+), 179 deletions(-)
>
> --
> 1.8.3.1
>
Ah, this series is incomplete.
Pls see the update one.
https://lore.kernel.org/bpf/20220810151840.16394-1-laoar.shao@gmail.com/T/#t
--
Regards
Yafang
Powered by blists - more mailing lists