netdev - Re: unregistering tcp_ca struct_ops can cause kernel page fault

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <09b0c78a-09be-474d-b080-bdd5e7c76e40@linux.dev>
Date: Mon, 9 Dec 2024 13:56:21 -0800
From: Martin KaFai Lau <martin.lau@...ux.dev>
To: rtm@...il.mit.edu
Cc: bpf@...r.kernel.org, netdev@...r.kernel.org
Subject: Re: unregistering tcp_ca struct_ops can cause kernel page fault

On 12/8/24 6:59 AM, rtm@...il.mit.edu wrote:
> The attached program, along with the attached bpf_cubic.o eBPF binary,
> uses bpftool to install a tcp_ca struct ops, creates a tcp connection,
> and then unregisters the struct ops while the tcp connection is still
> active. On my 6.13.0 system this causes a kernel fault due to

I tried the latest bpf-next/master (6.13-r2 + some recent bpf patches). I cannot 
reproduce. I also don't recall any recent fix related to struct_ops refcnt in 
the bpf_struct_ops or bpf_tcp_ca.

I don't see how this could happen now because the icsk->icsk_ca_ops is 
refcnt-ed. e.g. the setsockopt(TCP_CONGESTION) in your attached tcpbpf12a.c. 
bpf_try_module_get, which then calls bpf_struct_ops_get for the bpf written 
tcp-cc, should have refcnt-ed the icsk_ca_ops. I traced on the kernel code path 
triggered by your tcpbpf12a.c and the refcnt looks correct.

If you can reproduce consistently, please help to debug and narrow further why 
the bpf_struct_ops_map_free() cleanup is finished while a tcp_sock still has a 
hold of it in icsk->icsk_ca_ops.

> tcp_tso_segs() calling through a de-allocated struct tcp_congestion_ops.
> 
> bpf_cubic.o came from
> 
>    https://github.com/aroodgar/bpf-tcp-congestion-control-algorithm
> 
> Linux ubuntu66 6.13.0-rc1-00337-g7503345ac5f5 #11 SMP Sun Dec  8 08:37:57 EST 2024 x86_64 x86_64 x86_64 GNU/Linux
> 
> Oops: general protection fault, probably for non-canonical address 0x6b6b6b6b6b6b6b6b: 0000 [#1] SMP DEBUG_PAGEALLOC PTI
> CPU: 6 UID: 0 PID: 1594 Comm: a.out Not tainted 6.13.0-rc1-00337-g7503345ac5f5 #11
> Hardware name: FreeBSD BHYVE/BHYVE, BIOS 14.0 10/17/2021
> RIP: 0010:__x86_indirect_thunk_array+0xa/0x20
> Code: 66 0f 1f 00 31 ff e9 15 70 d6 fe cc cc cc cc cc cc cc cc cc cc cc cc cc cc
>   cc cc cc cc cc cc cc e8 01 00 00 00 cc 48 89 04 24 <c3> cc cc cc cc 90 66 66 2e
>   0f 1f 84 00 00 00 00 00 0f 1f 44 00 00
> RSP: 0018:ffffc90002037c60 EFLAGS: 00010202
> RAX: 6b6b6b6b6b6b6b6b RBX: ffff88810afc0b00 RCX: 0000000000000018
> RDX: 00000000077b668a RSI: 0000000000008000 RDI: ffff88810afc0b00
> RBP: 0000000000008000 R08: 0000000000000000 R09: 0000000000008000
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
> R13: 0000000000008000 R14: 0000000000000000 R15: ffffffff842f6140
> FS:  00007f5457b84740(0000) GS:ffff88842db80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 00007f5457d8c710 CR3: 000000011b4b8005 CR4: 00000000003706f0
> DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> Call Trace:
>   <TASK>
>   ? die_addr+0x31/0x80
>   ? exc_general_protection+0x1b4/0x3c0
>   ? asm_exc_general_protection+0x26/0x30
>   ? __x86_indirect_thunk_array+0xa/0x20
>   ? tcp_tso_segs+0x1c/0x90
>   ? tcp_write_xmit+0x74/0x1840
>   ? __mod_memcg_state+0x91/0x190
>   ? __tcp_push_pending_frames+0x31/0xc0
>   ? tcp_sendmsg_locked+0xafc/0xd10
>   ? tcp_sendmsg+0x26/0x40
>   ? sock_write_iter+0x167/0x1a0
>   ? vfs_write+0x35d/0x400
>   ? ksys_write+0xc6/0xe0
>   ? do_syscall_64+0x3f/0xd0
>   ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
>   </TASK>
> 
> Robert Morris
> rtm@....edu
>