linux-kernel - Re: [PATCH net-next 0/4] bpf: bpf_htab: Add BPF_MAP_TYPE_PERCPU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160112060007.GB31180@ast-mbp.thefacebook.com>
Date:	Mon, 11 Jan 2016 22:00:08 -0800
From:	Alexei Starovoitov <alexei.starovoitov@...il.com>
To:	Ming Lei <tom.leiming@...il.com>
Cc:	Martin KaFai Lau <kafai@...com>,
	Network Development <netdev@...r.kernel.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	FB Kernel Team <kernel-team@...com>
Subject: Re: [PATCH net-next 0/4] bpf: bpf_htab: Add BPF_MAP_TYPE_PERCPU_HASH

On Tue, Jan 12, 2016 at 01:48:10PM +0800, Ming Lei wrote:
> The total memory consumption is still much less than memory consumed
> by percpu hash since a new element is only added to hash if the key is run
> on that CPU. Most of times, for one key it may touch very few CPUs.
> 
> For percpu hash, the memory is always allocated to every CPU no matter
> if the key is run from the CPU.

In Martin's use case all cpus are servicing network traffic and all of them
are counting packets.

> In my test, removing the current kmalloc() in update element callback can
> improve io thoughput by 10% not mention the percpu ida allocation cost, and
> looks it isn't cheap. That is why I don't think it is good to
> introduce another new
> allocation in the eBPF prog path.

I don't think anyone is arguing that pre-allocation is not needed.
In some cases better performance can be achieved with pre-allocation,
in some other cases regular hash map will be enough,
and in others hash map with per-cpu is needed as well.

> You can find my test in the link below:
> 
>        https://patchwork.ozlabs.org/patch/556926/

yes, for tools/biolatency pre-allocation is a win,
but in many other cases we simply cannot pre-allocate all elements.