| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <586d63b4-1828-f633-a4ff-88e4e23d164a@fb.com>
Date: Fri, 20 Nov 2020 16:31:11 -0800
From: Yonghong Song <yhs@...com>
To: Weqaar Janjua <weqaar.janjua@...il.com>, <bpf@...r.kernel.org>,
<netdev@...r.kernel.org>, <daniel@...earbox.net>, <ast@...nel.org>,
<magnus.karlsson@...il.com>, <bjorn.topel@...el.com>
CC: Weqaar Janjua <weqaar.a.janjua@...el.com>, <shuah@...nel.org>,
<skhan@...uxfoundation.org>, <linux-kselftest@...r.kernel.org>,
<anders.roxell@...aro.org>, <jonathan.lemon@...il.com>
Subject: Re: [PATCH bpf-next v2 0/5] selftests/bpf: xsk selftests
On 11/20/20 5:00 AM, Weqaar Janjua wrote:
> This patch set adds AF_XDP selftests based on veth to selftests/bpf.
>
> # Topology:
> # ---------
> # -----------
> # _ | Process | _
> # / ----------- \
> # / | \
> # / | \
> # ----------- | -----------
> # | Thread1 | | | Thread2 |
> # ----------- | -----------
> # | | |
> # ----------- | -----------
> # | xskX | | | xskY |
> # ----------- | -----------
> # | | |
> # ----------- | ----------
> # | vethX | --------- | vethY |
> # ----------- peer ----------
> # | | |
> # namespaceX | namespaceY
>
> These selftests test AF_XDP SKB and Native/DRV modes using veth Virtual
> Ethernet interfaces.
>
> The test program contains two threads, each thread is single socket with
> a unique UMEM. It validates in-order packet delivery and packet content
> by sending packets to each other.
>
> Prerequisites setup by script test_xsk_prerequisites.sh:
>
> Set up veth interfaces as per the topology shown ^^:
> * setup two veth interfaces and one namespace
> ** veth<xxxx> in root namespace
> ** veth<yyyy> in af_xdp<xxxx> namespace
> ** namespace af_xdp<xxxx>
> * create a spec file veth.spec that includes this run-time configuration
> that is read by test scripts - filenames prefixed with test_xsk_
> *** xxxx and yyyy are randomly generated 4 digit numbers used to avoid
> conflict with any existing interface
>
> The following tests are provided:
>
> 1. AF_XDP SKB mode
> Generic mode XDP is driver independent, used when the driver does
> not have support for XDP. Works on any netdevice using sockets and
> generic XDP path. XDP hook from netif_receive_skb().
> a. nopoll - soft-irq processing
> b. poll - using poll() syscall
> c. Socket Teardown
> Create a Tx and a Rx socket, Tx from one socket, Rx on another.
> Destroy both sockets, then repeat multiple times. Only nopoll mode
> is used
> d. Bi-directional Sockets
> Configure sockets as bi-directional tx/rx sockets, sets up fill
> and completion rings on each socket, tx/rx in both directions.
> Only nopoll mode is used
>
> 2. AF_XDP DRV/Native mode
> Works on any netdevice with XDP_REDIRECT support, driver dependent.
> Processes packets before SKB allocation. Provides better performance
> than SKB. Driver hook available just after DMA of buffer descriptor.
> a. nopoll
> b. poll
> c. Socket Teardown
> d. Bi-directional Sockets
> * Only copy mode is supported because veth does not currently support
> zero-copy mode
>
> Total tests: 8
>
> Flow:
> * Single process spawns two threads: Tx and Rx
> * Each of these two threads attach to a veth interface within their
> assigned namespaces
> * Each thread creates one AF_XDP socket connected to a unique umem
> for each veth interface
> * Tx thread transmits 10k packets from veth<xxxx> to veth<yyyy>
> * Rx thread verifies if all 10k packets were received and delivered
> in-order, and have the right content
>
> v2 changes:
> * Move selftests/xsk to selftests/bpf
> * Remove Makefiles under selftests/xsk, and utilize selftests/bpf/Makefile
>
> Structure of the patch set:
>
> Patch 1: This patch adds XSK Selftests framework under selftests/bpf
> Patch 2: Adds tests: SKB poll and nopoll mode, and mac-ip-udp debug
> Patch 3: Adds tests: DRV poll and nopoll mode
> Patch 4: Adds tests: SKB and DRV Socket Teardown
> Patch 5: Adds tests: SKB and DRV Bi-directional Sockets
I just want to report that after applying the above 5 patches
on top of bpf-next commit 450d060e8f75 ("bpftool: Add {i,d}tlb_misses
support for bpftool profile"), I hit the following error with below
command sequences:
$ ./test_xsk_prerequisites.sh
$ ./test_xsk_skb_poll.sh
# Interface found: ve1480
# Interface found: ve9258
# NS switched: af_xdp9258
1..1
# Interface [ve9258] vector [Rx]
# Interface [ve1480] vector [Tx]
# Sending 10000 packets on interface ve1480
[ 331.741244] ------------[ cut here ]------------
[ 331.741741] kernel BUG at net/core/skbuff.c:1621!
[ 331.742265] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 331.742837] CPU: 0 PID: 1883 Comm: xdpxceiver Not tainted 5.10.0-rc3+
#1037
[ 331.743468] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS 1.9.3
-1.el7.centos 04/01/2014
[ 331.744300] RIP: 0010:pskb_expand_head+0x27b/0x310
[ 331.744747] Code: df e8 69 fc ff ff e9 ab fe ff ff 44 2b 6c 24 04 44
01 ab d0
00 00 00 48 83 c4 08 31 c0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 0b <0f>
0b be 02
00 00 00 e8 89 4e ad ff 8b 83 bc 00 00 00 48 03 83 c0
[ 331.746414] RSP: 0018:ffffbae4c0003d08 EFLAGS: 00010202
[ 331.746892] RAX: 000000000000013f RBX: ffff9e0a8367ad00 RCX:
0000000000000a20
[ 331.747534] RDX: 0000000000000002 RSI: 0000000000000100 RDI:
ffff9e0a8367ad00
[ 331.748192] RBP: ffffbae4c00b2000 R08: 0000000000000001 R09:
000000000000000e
[ 331.748834] R10: ffffbae4c0003eb8 R11: 00000000ef974e19 R12:
ffff9e0a86ecf000
[ 331.749472] R13: 0000000000000001 R14: ffff9e0a8367ad00 R15:
ffff9e0a8367ad00
[ 331.750119] FS: 00007ff0806c5e00(0000) GS:ffff9e0abae00000(0000)
knlGS:00000
00000000000
[ 331.750848] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 331.751379] CR2: 00007ff0806c01d8 CR3: 0000000106e00006 CR4:
0000000000370ef0
[ 331.752022] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 331.752665] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 331.753307] Call Trace:
[ 331.753535] <IRQ>
[ 331.753736] do_xdp_generic.part.157+0xa3/0x550
[ 331.754151] __netif_receive_skb_core+0x67e/0x12b0
[ 331.754588] ? process_backlog+0x86/0x250
[ 331.754961] ? __netif_receive_skb_one_core+0x3c/0xa0
[ 331.755419] __netif_receive_skb_one_core+0x3c/0xa0
[ 331.755865] process_backlog+0xf5/0x250
[ 331.756215] net_rx_action+0x144/0x440
[ 331.756559] __do_softirq+0xe4/0x493
[ 331.756894] asm_call_irq_on_stack+0x12/0x20
[ 331.757282] </IRQ>
[ 331.757478] ? dev_direct_xmit+0x1e8/0x230
[ 331.757856] do_softirq_own_stack+0x81/0xa0
[ 331.758244] do_softirq.part.16+0x3c/0x80
[ 331.758611] __local_bh_enable_ip+0xda/0xe0
[ 331.758995] dev_direct_xmit+0x20d/0x230
[ 331.759356] __xsk_sendmsg+0x314/0x3d0
[ 331.759704] sock_sendmsg+0x5b/0x60
[ 331.760025] __sys_sendto+0xf1/0x160
[ 331.760355] ? lockdep_hardirqs_on+0xbf/0x130
[ 331.760759] ? syscall_enter_from_user_mode+0x1c/0x50
[ 331.761216] __x64_sys_sendto+0x24/0x30
[ 331.761563] do_syscall_64+0x33/0x40
[ 331.761895] entry_SYSCALL_64_after_hwframe+0x44/0xa9
[ 331.762357] RIP: 0033:0x7ff08c8b9633
[ 331.762689] Code: 49 89 ca b8 2c 00 00 00 0f 05 48 3d 01 f0 ff ff 73
34 c3 48
83 ec 08 e8 1b f7 ff ff 48 89 04 24 49 89 ca b8 2c 00 00 00 0f 05 <48>
8b 3c 24
48 89 c2 e8 61 f7 ff ff 48 89 d0 48 83 c4 08 48 3d 01
[ 331.764356] RSP: 002b:00007ff0806c55c0 EFLAGS: 00000293 ORIG_RAX:
00000000000
0002c
[ 331.765038] RAX: ffffffffffffffda RBX: 00007ff0780009b0 RCX:
00007ff08c8b9633
[ 331.765684] RDX: 0000000000000000 RSI: 0000000000000000 RDI:
0000000000000008
[ 331.766324] RBP: 0000000000000040 R08: 0000000000000000 R09:
0000000000000000
[ 331.766969] R10: 0000000000000040 R11: 0000000000000293 R12:
0000000000000040
[ 331.767608] R13: 0000000000000040 R14: 0000000000000000 R15:
0000000000cd4030
[ 331.768261] Modules linked in:
[ 331.768596] ---[ end trace d9ca37a7957928dd ]---
[ 331.769126] RIP: 0010:pskb_expand_head+0x27b/0x310
[ 331.769678] Code: df e8 69 fc ff ff e9 ab fe ff ff 44 2b 6c 24 04 44
01 ab d0
00 00 00 48 83 c4 08 31 c0 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 0b <0f>
0b be 02
00 00 00 e8 89 4e ad ff 8b 83 bc 00 00 00 48 03 83 c0
[ 331.771459] RSP: 0018:ffffbae4c0003d08 EFLAGS: 00010202
[ 331.772043] RAX: 000000000000013f RBX: ffff9e0a8367ad00 RCX:
0000000000000a20
[ 331.772784] RDX: 0000000000000002 RSI: 0000000000000100 RDI:
ffff9e0a8367ad00
[ 331.773526] RBP: ffffbae4c00b2000 R08: 0000000000000001 R09:
000000000000000e
[ 331.774293] R10: ffffbae4c0003eb8 R11: 00000000ef974e19 R12:
ffff9e0a86ecf000
[ 331.775049] R13: 0000000000000001 R14: ffff9e0a8367ad00 R15:
ffff9e0a8367ad00
[ 331.775901] FS: 00007ff0806c5e00(0000) GS:ffff9e0abae00000(0000)
knlGS:00000
00000000000
[ 331.776809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 331.777455] CR2: 00007ff0806c01d8 CR3: 0000000106e00006 CR4:
0000000000370ef0
[ 331.778232] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 331.778989] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7:
0000000000000400
[ 331.779739] Kernel panic - not syncing: Fatal exception in interrupt
[ 331.780523] Kernel Offset: 0xb800000 from 0xffffffff81000000
(relocation rang
e: 0xffffffff80000000-0xffffffffbfffffff)
[ 331.781488] ---[ end Kernel panic - not syncing: Fatal exception in
interrupt
]---
In any case, kernel should not panic. You or somebody familiar with xsk
may want to take a look.
>
> Thanks: Weqaar
>
> Weqaar Janjua (5):
> selftests/bpf: xsk selftests framework
> selftests/bpf: xsk selftests - SKB POLL, NOPOLL
> selftests/bpf: xsk selftests - DRV POLL, NOPOLL
> selftests/bpf: xsk selftests - Socket Teardown - SKB, DRV
> selftests/bpf: xsk selftests - Bi-directional Sockets - SKB, DRV
>
> tools/testing/selftests/bpf/Makefile | 15 +-
> .../bpf/test_xsk_drv_bidirectional.sh | 23 +
> .../selftests/bpf/test_xsk_drv_nopoll.sh | 20 +
> .../selftests/bpf/test_xsk_drv_poll.sh | 20 +
> .../selftests/bpf/test_xsk_drv_teardown.sh | 20 +
> .../selftests/bpf/test_xsk_prerequisites.sh | 127 ++
> .../bpf/test_xsk_skb_bidirectional.sh | 20 +
> .../selftests/bpf/test_xsk_skb_nopoll.sh | 20 +
> .../selftests/bpf/test_xsk_skb_poll.sh | 20 +
> .../selftests/bpf/test_xsk_skb_teardown.sh | 20 +
> tools/testing/selftests/bpf/xdpxceiver.c | 1056 +++++++++++++++++
> tools/testing/selftests/bpf/xdpxceiver.h | 158 +++
> tools/testing/selftests/bpf/xsk_env.sh | 28 +
> tools/testing/selftests/bpf/xsk_prereqs.sh | 119 ++
> 14 files changed, 1664 insertions(+), 2 deletions(-)
[...]
Powered by blists - more mailing lists