Date:   Tue,  8 Mar 2022 13:10:47 +0000
From:   Yafang Shao <laoar.shao@...il.com>
To:     ast@...nel.org, daniel@...earbox.net, andrii@...nel.org,
        kafai@...com, songliubraving@...com, yhs@...com,
        john.fastabend@...il.com, kpsingh@...nel.org,
        akpm@...ux-foundation.org, cl@...ux.com, penberg@...nel.org,
        rientjes@...gle.com, iamjoonsoo.kim@....com, vbabka@...e.cz,
        hannes@...xchg.org, mhocko@...nel.org, vdavydov.dev@...il.com,
        guro@...com
Cc:     linux-mm@...ck.org, netdev@...r.kernel.org, bpf@...r.kernel.org,
        Yafang Shao <laoar.shao@...il.com>
Subject: [PATCH RFC 0/9] bpf, mm: recharge bpf memory from offline memcg

When we use memcg to limit the containers which load bpf progs and maps,
we find there is an issue that the lifecycle of container and bpf are not
always the same, because we may pin the maps and progs while update the
container only. So once the container which has alreay pinned progs and
maps is restarted, the pinned progs and maps are no longer charged to it
any more. In other words, this kind of container can steal memory from the
host, that is not expected by us. This patchset means to resolve this
issue.

After the container is restarted, the old memcg which is charged by the
pinned progs and maps will be offline but won't be freed until all of the
related maps and progs are freed. If we want to charge these bpf memory to
the new started memcg, we should uncharge them from the offline memcg first
and then charge it to the new one. As we have already known how the bpf
memroy is allocated and freed, we can also know how to charge and uncharge
it. This pathset implements various charge and uncharge methords for these
memory.

Regarding how to do the recharge, we decide to implement new bpf syscalls
to do it. With the new implemented bpf syscall, the agent running in the
container can use it to do the recharge. As of now we only implement it for
the bpf hash maps. Below is a simple example how to do the recharge,

====
int main(int argc, char *argv[])
{
	union bpf_attr attr = {};
	int map_id;
	int pfd;

	if (argc < 2) {
		printf("Pls. give a map id \n");
		exit(-1);
	}

	map_id = atoi(argv[1]);
	attr.map_id = map_id;
	pfd = syscall(SYS_bpf, BPF_MAP_RECHARGE, &attr, sizeof(attr));
	if (pfd < 0)
		perror("BPF_MAP_RECHARGE");

	return 0;
}

====

Patch #1 and #2 is for the observability, with which we can easily check
whether the bpf maps is charged to a memcg and whether the memcg is offline.
Patch #3, #4 and #5 is for the charge and uncharge methord for vmalloc-ed,
kmalloc-ed and percpu memory.
Patch #6~#9 implements the recharge of bpf hash map, which is mostly used
by our bpf services. The other maps hasn't been implemented yet. The bpf progs
hasn't been implemented neither.

This pathset is still a POC now, with limited testing. Any feedback is
welcomed.

Yafang Shao (9):
  bpftool: fix print error when show bpf man
  bpftool: show memcg info of bpf map
  mm: add methord to charge kmalloc-ed address
  mm: add methord to charge vmalloc-ed address
  mm: add methord to charge percpu address
  bpf: add a helper to find map by id
  bpf: add BPF_MAP_RECHARGE syscall
  bpf: make bpf_map_{save, release}_memcg public
  bpf: support recharge for hash map

 include/linux/bpf.h            | 23 +++++++++++++
 include/linux/percpu.h         |  1 +
 include/linux/slab.h           |  2 ++
 include/linux/vmalloc.h        |  1 +
 include/uapi/linux/bpf.h       | 10 ++++++
 kernel/bpf/hashtab.c           | 35 ++++++++++++++++++++
 kernel/bpf/syscall.c           | 73 ++++++++++++++++++++++++++----------------
 mm/percpu.c                    | 50 +++++++++++++++++++++++++++++
 mm/slab.c                      |  6 ++++
 mm/slob.c                      |  6 ++++
 mm/slub.c                      | 32 ++++++++++++++++++
 mm/util.c                      |  9 ++++++
 mm/vmalloc.c                   | 29 +++++++++++++++++
 tools/bpf/bpftool/map.c        |  9 +++---
 tools/include/uapi/linux/bpf.h |  1 +
 15 files changed, 254 insertions(+), 33 deletions(-)

-- 
1.8.3.1

Powered by blists - more mailing lists