netdev - Re: [PATCH v7 bpf-next 0/9] introduce support for XDP programs in CPUMAP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200717171333.3fe979e6@toad>
Date:   Fri, 17 Jul 2020 17:13:33 +0200
From:   Jakub Sitnicki <jakub@...udflare.com>
To:     Lorenzo Bianconi <lorenzo@...nel.org>
Cc:     netdev@...r.kernel.org, davem@...emloft.net, ast@...nel.org,
        brouer@...hat.com, daniel@...earbox.net, toke@...hat.com,
        lorenzo.bianconi@...hat.com, dsahern@...nel.org,
        andrii.nakryiko@...il.com, bpf@...r.kernel.org
Subject: Re: [PATCH v7 bpf-next 0/9] introduce support for XDP programs in
 CPUMAP

On Fri, 17 Jul 2020 13:01:36 +0200
Lorenzo Bianconi <lorenzo@...nel.org> wrote:

> [...]
> 
> > This started showing up with when running ./test_progs from recent
> > bpf-next (bfdfa51702de). Any chance it is related?
> > 
> > [ 2950.440613] =============================================
> > 
> > [ 3073.281578] INFO: task cpumap/0/map:26:536 blocked for more than 860 seconds.
> > [ 3073.285492]       Tainted: G        W         5.8.0-rc4-01471-g15d51f3a516b #814
> > [ 3073.289177] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> > [ 3073.293021] cpumap/0/map:26 D    0   536      2 0x00004000
> > [ 3073.295755] Call Trace:
> > [ 3073.297143]  __schedule+0x5ad/0xf10
> > [ 3073.299032]  ? pci_mmcfg_check_reserved+0xd0/0xd0
> > [ 3073.301416]  ? static_obj+0x31/0x80
> > [ 3073.303277]  ? mark_held_locks+0x24/0x90
> > [ 3073.305313]  ? cpu_map_update_elem+0x6d0/0x6d0
> > [ 3073.307544]  schedule+0x6f/0x160
> > [ 3073.309282]  schedule_preempt_disabled+0x14/0x20
> > [ 3073.311593]  kthread+0x175/0x240
> > [ 3073.313299]  ? kthread_create_on_node+0xd0/0xd0
> > [ 3073.315106]  ret_from_fork+0x1f/0x30
> > [ 3073.316365]
> >                Showing all locks held in the system:
> > [ 3073.318423] 1 lock held by khungtaskd/33:
> > [ 3073.319642]  #0: ffffffff82d246a0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x28/0x1c3
> > 
> > [ 3073.322249] =============================================  
> 
> Hi Jakub,
> 
> can you please provide more info? can you please identify the test that trigger
> the issue? I run test_progs with bpf-next master branch and it works fine for me.
> I run the tests in a vm with 4 vcpus and 4G of memory.
> 
> Regards,
> Lorenzo
> 

Was able to trigger it running the newly added selftest:

virtme-init: console is ttyS0
bash-5.0# ./test_progs -n 100
#100/1 cpumap_with_progs:OK
#100 xdp_cpumap_attach:OK
Summary: 1/1 PASSED, 0 SKIPPED, 0 FAILED
bash-5.0# [  247.177168] INFO: task cpumap/0/map:3:198 blocked for more than 122 seconds.
[  247.181306]       Not tainted 5.8.0-rc4-01456-gbfdfa51702de #815
[  247.184487] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[  247.188876] cpumap/0/map:3  D    0   198      2 0x00004000
[  247.192624] Call Trace:
[  247.194327]  __schedule+0x5ad/0xf10
[  247.196860]  ? pci_mmcfg_check_reserved+0xd0/0xd0
[  247.199853]  ? static_obj+0x31/0x80
[  247.201917]  ? mark_held_locks+0x24/0x90
[  247.204398]  ? cpu_map_update_elem+0x6d0/0x6d0
[  247.207098]  schedule+0x6f/0x160
[  247.209079]  schedule_preempt_disabled+0x14/0x20
[  247.211863]  kthread+0x175/0x240
[  247.213698]  ? kthread_create_on_node+0xd0/0xd0
[  247.216054]  ret_from_fork+0x1f/0x30
[  247.218363]
[  247.218363] Showing all locks held in the system:
[  247.222150] 1 lock held by khungtaskd/33:
[  247.224894]  #0: ffffffff82d246a0 (rcu_read_lock){....}-{1:2}, at: debug_show_all_locks+0x28/0x1c3
[  247.231113]
[  247.232335] =============================================
[  247.232335]

qemu running with 4 vCPUs, 4 GB of memory. .config uploaded at
https://paste.centos.org/view/0c14663d

HTH,
-jkbs