| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <0ed19302-a43c-c04e-110e-eb1f0a72146f@fb.com>
Date: Wed, 15 Jan 2020 20:13:07 +0000
From: Yonghong Song <yhs@...com>
To: Brian Vazquez <brianvv@...gle.com>,
Brian Vazquez <brianvv.kernel@...il.com>,
Alexei Starovoitov <ast@...nel.org>,
"Daniel Borkmann" <daniel@...earbox.net>,
"David S . Miller" <davem@...emloft.net>
CC: Andrii Nakryiko <andrii.nakryiko@...il.com>,
Stanislav Fomichev <sdf@...gle.com>,
Petar Penkov <ppenkov@...gle.com>,
Willem de Bruijn <willemb@...gle.com>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
"bpf@...r.kernel.org" <bpf@...r.kernel.org>
Subject: Re: [PATCH v5 bpf-next 0/9] add bpf batch ops to process more than 1
elem
On 1/15/20 10:42 AM, Brian Vazquez wrote:
> This patch series introduce batch ops that can be added to bpf maps to
> lookup/lookup_and_delete/update/delete more than 1 element at the time,
> this is specially useful when syscall overhead is a problem and in case
> of hmap it will provide a reliable way of traversing them.
>
> The implementation inclues a generic approach that could potentially be
> used by any bpf map and adds it to arraymap, it also includes the specific
> implementation of hashmaps which are traversed using buckets instead
> of keys.
>
> The bpf syscall subcommands introduced are:
>
> BPF_MAP_LOOKUP_BATCH
> BPF_MAP_LOOKUP_AND_DELETE_BATCH
> BPF_MAP_UPDATE_BATCH
> BPF_MAP_DELETE_BATCH
>
> The UAPI attribute is:
>
> struct { /* struct used by BPF_MAP_*_BATCH commands */
> __aligned_u64 in_batch; /* start batch,
> * NULL to start from beginning
> */
> __aligned_u64 out_batch; /* output: next start batch */
> __aligned_u64 keys;
> __aligned_u64 values;
> __u32 count; /* input/output:
> * input: # of key/value
> * elements
> * output: # of filled elements
> */
> __u32 map_fd;
> __u64 elem_flags;
> __u64 flags;
> } batch;
>
>
> in_batch and out_batch are only used for lookup and lookup_and_delete since
> those are the only two operations that attempt to traverse the map.
>
> update/delete batch ops should provide the keys/values that user wants
> to modify.
>
> Here are the previous discussions on the batch processing:
> - https://lore.kernel.org/bpf/20190724165803.87470-1-brianvv@google.com/
> - https://lore.kernel.org/bpf/20190829064502.2750303-1-yhs@fb.com/
> - https://lore.kernel.org/bpf/20190906225434.3635421-1-yhs@fb.com/
>
> Changelog sinve v4:
> - Remove unnecessary checks from libbpf API (Andrii Nakryiko)
> - Move DECLARE_LIBBPF_OPTS with all var declarations (Andrii Nakryiko)
> - Change bucket internal buffer size to 5 entries (Yonghong Song)
> - Fix some minor bugs in hashtab batch ops implementation (Yonghong Song)
>
> Changelog sinve v3:
> - Do not use copy_to_user inside atomic region (Yonghong Song)
> - Use _opts approach on libbpf APIs (Andrii Nakryiko)
> - Drop generic_map_lookup_and_delete_batch support
> - Free malloc-ed memory in tests (Yonghong Song)
> - Reverse christmas tree (Yonghong Song)
> - Add acked labels
>
> Changelog sinve v2:
> - Add generic batch support for lpm_trie and test it (Yonghong Song)
> - Use define MAP_LOOKUP_RETRIES for retries (John Fastabend)
> - Return errors directly and remove labels (Yonghong Song)
> - Insert new API functions into libbpf alphabetically (Yonghong Song)
> - Change hlist_nulls_for_each_entry_rcu to
> hlist_nulls_for_each_entry_safe in htab batch ops (Yonghong Song)
>
> Changelog since v1:
> - Fix SOB ordering and remove Co-authored-by tag (Alexei Starovoitov)
>
> Changelog since RFC:
> - Change batch to in_batch and out_batch to support more flexible opaque
> values to iterate the bpf maps.
> - Remove update/delete specific batch ops for htab and use the generic
> implementations instead.
>
> Brian Vazquez (5):
> bpf: add bpf_map_{value_size,update_value,map_copy_value} functions
> bpf: add generic support for lookup batch op
> bpf: add generic support for update and delete batch ops
> bpf: add lookup and update batch ops to arraymap
> selftests/bpf: add batch ops testing to array bpf map
>
> Yonghong Song (4):
> bpf: add batch ops to all htab bpf map
> tools/bpf: sync uapi header bpf.h
> libbpf: add libbpf support to batch ops
> selftests/bpf: add batch ops testing for htab and htab_percpu map
>
> include/linux/bpf.h | 18 +
> include/uapi/linux/bpf.h | 21 +
> kernel/bpf/arraymap.c | 2 +
> kernel/bpf/hashtab.c | 264 +++++++++
> kernel/bpf/syscall.c | 554 ++++++++++++++----
> tools/include/uapi/linux/bpf.h | 21 +
> tools/lib/bpf/bpf.c | 58 ++
> tools/lib/bpf/bpf.h | 22 +
> tools/lib/bpf/libbpf.map | 4 +
> .../bpf/map_tests/array_map_batch_ops.c | 129 ++++
> .../bpf/map_tests/htab_map_batch_ops.c | 283 +++++++++
> 11 files changed, 1248 insertions(+), 128 deletions(-)
> create mode 100644 tools/testing/selftests/bpf/map_tests/array_map_batch_ops.c
> create mode 100644 tools/testing/selftests/bpf/map_tests/htab_map_batch_ops.c
Thanks for the work! LGTM. Ack for the whole series.
Acked-by: Yonghong Song <yhs@...com>
Powered by blists - more mailing lists