Message-ID: <d1a7dea8-ce20-4c6f-beed-8a28b07e9468@ovn.org>
Date: Tue, 3 Dec 2024 12:58:31 +0100
From: Ilya Maximets <i.maximets@....org>
To: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Cc: i.maximets@....org, Jakub Kicinski <kuba@...nel.org>,
 "David S. Miller" <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>,
 Paolo Abeni <pabeni@...hat.com>, LKML <linux-kernel@...r.kernel.org>
Subject: [v6.12] BUG: KASAN: slab-use-after-free in dst_destroy+0x2e2/0x340

Hello there.  I was running some tests with openvswitch+ipsec on v6.12 tag
and got the KASAN UAF splat provided below.  It doesn't seem to be related
to anything specific to openvswitch module, more like core parts of networking.
At lest, at the first glance.

For the context, what I'm running is an OVS system test that creates 20 network
namespaces, starts OVS and Libreswan in each of them, creates a full mesh of
Geneve tunnels with IPsec (a separate tunnel between each pair of namespaces),
then checks that pings work through all the tunnels and then deletes all the
ports, OVS datapath and namespaces.  While removing namespaces, I see the
following KASAN report in the logs:

Dec 03 05:46:17 kernel: genev_sys_6081 (unregistering): left promiscuous mode
Dec 03 05:46:17 kernel: br-ipsec: left promiscuous mode
Dec 03 05:46:17 kernel: ovs-system: left promiscuous mode
Dec 03 05:46:18 kernel: ==================================================================
Dec 03 05:46:18 kernel: BUG: KASAN: slab-use-after-free in dst_destroy+0x2e2/0x340
Dec 03 05:46:18 kernel: Read of size 8 at addr ffff8882137ccab0 by task swapper/37/0
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: CPU: 37 UID: 0 PID: 0 Comm: swapper/37 Kdump: loaded Not tainted 6.12.0 #67
Dec 03 05:46:18 kernel: Hardware name: Red Hat KVM/RHEL, BIOS 1.16.1-1.el9 04/01/2014
Dec 03 05:46:18 kernel: Call Trace:
Dec 03 05:46:18 kernel:  <IRQ>
Dec 03 05:46:18 kernel:  dump_stack_lvl+0x64/0xa0
Dec 03 05:46:18 kernel:  print_address_description.constprop.0+0x2c/0x3d0
Dec 03 05:46:18 kernel:  ? dst_destroy+0x2e2/0x340
Dec 03 05:46:18 kernel:  print_report+0xb4/0x270
Dec 03 05:46:18 kernel:  ? dst_destroy+0x2e2/0x340
Dec 03 05:46:18 kernel:  ? kasan_addr_to_slab+0x9/0xa0
Dec 03 05:46:18 kernel:  kasan_report+0x89/0xc0
Dec 03 05:46:18 kernel:  ? dst_destroy+0x2e2/0x340
Dec 03 05:46:18 kernel:  ? rcu_do_batch+0x377/0xeb0
Dec 03 05:46:18 kernel:  dst_destroy+0x2e2/0x340
Dec 03 05:46:18 kernel:  rcu_do_batch+0x379/0xeb0
Dec 03 05:46:18 kernel:  ? __pfx_rcu_do_batch+0x10/0x10
Dec 03 05:46:18 kernel:  ? lockdep_hardirqs_on_prepare+0x127/0x3e0
Dec 03 05:46:18 kernel:  rcu_core+0x354/0x510
Dec 03 05:46:18 kernel:  handle_softirqs+0x1fe/0x580
Dec 03 05:46:18 kernel:  __irq_exit_rcu+0x13a/0x190
Dec 03 05:46:18 kernel:  irq_exit_rcu+0xa/0x20
Dec 03 05:46:18 kernel:  sysvec_apic_timer_interrupt+0x72/0x90
Dec 03 05:46:18 kernel:  </IRQ>
Dec 03 05:46:18 kernel:  <TASK>
Dec 03 05:46:18 kernel:  asm_sysvec_apic_timer_interrupt+0x16/0x20
Dec 03 05:46:18 kernel: RIP: 0010:default_idle+0xb/0x20
Dec 03 05:46:18 kernel: Code: 00 4d 29 c8 4c 01 c7 4c 29 c2 e9 6e ff ff ff 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 90 0f 00 2d c7 c9 27 00 fb f4 <fa> c3 cc cc cc cc 66 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 40 00 90
Dec 03 05:46:18 kernel: RSP: 0018:ffff888100d2fe00 EFLAGS: 00000246
Dec 03 05:46:18 kernel: RAX: 00000000001870ed RBX: 1ffff110201a5fc2 RCX: ffffffffb61a3e46
Dec 03 05:46:18 kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffb3d4d123
Dec 03 05:46:18 kernel: RBP: 0000000000000000 R08: 0000000000000001 R09: ffffed11c7e1835d
Dec 03 05:46:18 kernel: R10: ffff888e3f0c1aeb R11: 0000000000000000 R12: 0000000000000000
Dec 03 05:46:18 kernel: R13: ffff888100d20000 R14: dffffc0000000000 R15: 0000000000000000
Dec 03 05:46:18 kernel:  ? ct_kernel_exit.constprop.0+0xb6/0xf0
Dec 03 05:46:18 kernel:  ? cpuidle_idle_call+0x1e3/0x270
Dec 03 05:46:18 kernel:  default_idle_call+0x67/0xa0
Dec 03 05:46:18 kernel:  cpuidle_idle_call+0x1e3/0x270
Dec 03 05:46:18 kernel:  ? __pfx_cpuidle_idle_call+0x10/0x10
Dec 03 05:46:18 kernel:  ? lock_release+0xd3/0x130
Dec 03 05:46:18 kernel:  ? lockdep_hardirqs_on_prepare+0x275/0x3e0
Dec 03 05:46:18 kernel:  ? tsc_verify_tsc_adjust+0x56/0x290
Dec 03 05:46:18 kernel:  do_idle+0xf1/0x1a0
Dec 03 05:46:18 kernel:  cpu_startup_entry+0x50/0x60
Dec 03 05:46:18 kernel:  start_secondary+0x210/0x290
Dec 03 05:46:18 kernel:  ? __pfx_start_secondary+0x10/0x10
Dec 03 05:46:18 kernel:  ? soft_restart_cpu+0x14/0x14
Dec 03 05:46:18 kernel:  common_startup_64+0x13e/0x141
Dec 03 05:46:18 kernel:  </TASK>
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: Allocated by task 12184:
Dec 03 05:46:18 kernel:  kasan_save_stack+0x20/0x40
Dec 03 05:46:18 kernel:  kasan_save_track+0x10/0x30
Dec 03 05:46:18 kernel:  __kasan_slab_alloc+0x83/0x90
Dec 03 05:46:18 kernel:  kmem_cache_alloc_noprof+0x123/0x3a0
Dec 03 05:46:18 kernel:  copy_net_ns+0xc2/0x530
Dec 03 05:46:18 kernel:  create_new_namespaces+0x35f/0x920
Dec 03 05:46:18 kernel:  unshare_nsproxy_namespaces+0x86/0x1b0
Dec 03 05:46:18 kernel:  ksys_unshare+0x2c0/0x6e0
Dec 03 05:46:18 kernel:  __x64_sys_unshare+0x2d/0x40
Dec 03 05:46:18 kernel:  do_syscall_64+0x8a/0x170
Dec 03 05:46:18 kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: Freed by task 11:
Dec 03 05:46:18 kernel:  kasan_save_stack+0x20/0x40
Dec 03 05:46:18 kernel:  kasan_save_track+0x10/0x30
Dec 03 05:46:18 kernel:  kasan_save_free_info+0x37/0x60
Dec 03 05:46:18 kernel:  __kasan_slab_free+0x50/0x70
Dec 03 05:46:18 kernel:  kmem_cache_free+0x1b8/0x560
Dec 03 05:46:18 kernel:  cleanup_net+0x767/0xa20
Dec 03 05:46:18 kernel:  process_one_work+0xe11/0x1640
Dec 03 05:46:18 kernel:  worker_thread+0x54d/0xc90
Dec 03 05:46:18 kernel:  kthread+0x2a8/0x380
Dec 03 05:46:18 kernel:  ret_from_fork+0x2d/0x70
Dec 03 05:46:18 kernel:  ret_from_fork_asm+0x1a/0x30
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: Last potentially related work creation:
Dec 03 05:46:18 kernel:  kasan_save_stack+0x20/0x40
Dec 03 05:46:18 kernel:  __kasan_record_aux_stack+0xad/0xc0
Dec 03 05:46:18 kernel:  insert_work+0x29/0x1b0
Dec 03 05:46:18 kernel:  __queue_work+0x5be/0x9c0
Dec 03 05:46:18 kernel:  queue_work_on+0x78/0x80
Dec 03 05:46:18 kernel:  xfrm_policy_insert+0x52f/0x6c0
Dec 03 05:46:18 kernel:  xfrm_add_policy+0x2a1/0x700
Dec 03 05:46:18 kernel:  xfrm_user_rcv_msg+0x4e5/0x830
Dec 03 05:46:18 kernel:  netlink_rcv_skb+0x12b/0x390
Dec 03 05:46:18 kernel:  xfrm_netlink_rcv+0x70/0x90
Dec 03 05:46:18 kernel:  netlink_unicast+0x447/0x710
Dec 03 05:46:18 kernel:  netlink_sendmsg+0x761/0xc40
Dec 03 05:46:18 kernel:  sock_write_iter+0x448/0x530
Dec 03 05:46:18 kernel:  vfs_write+0xa21/0xf30
Dec 03 05:46:18 kernel:  ksys_write+0x176/0x1d0
Dec 03 05:46:18 kernel:  do_syscall_64+0x8a/0x170
Dec 03 05:46:18 kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: Second to last potentially related work creation:
Dec 03 05:46:18 kernel:  kasan_save_stack+0x20/0x40
Dec 03 05:46:18 kernel:  __kasan_record_aux_stack+0xad/0xc0
Dec 03 05:46:18 kernel:  insert_work+0x29/0x1b0
Dec 03 05:46:18 kernel:  __queue_work+0x5be/0x9c0
Dec 03 05:46:18 kernel:  queue_work_on+0x78/0x80
Dec 03 05:46:18 kernel:  __xfrm_state_insert+0x179a/0x24b0
Dec 03 05:46:18 kernel:  xfrm_state_update+0x9ca/0xc60
Dec 03 05:46:18 kernel:  xfrm_add_sa+0x1b6/0x3e0
Dec 03 05:46:18 kernel:  xfrm_user_rcv_msg+0x4e5/0x830
Dec 03 05:46:18 kernel:  netlink_rcv_skb+0x12b/0x390
Dec 03 05:46:18 kernel:  xfrm_netlink_rcv+0x70/0x90
Dec 03 05:46:18 kernel:  netlink_unicast+0x447/0x710
Dec 03 05:46:18 kernel:  netlink_sendmsg+0x761/0xc40
Dec 03 05:46:18 kernel:  sock_write_iter+0x448/0x530
Dec 03 05:46:18 kernel:  vfs_write+0xa21/0xf30
Dec 03 05:46:18 kernel:  ksys_write+0x176/0x1d0
Dec 03 05:46:18 kernel:  do_syscall_64+0x8a/0x170
Dec 03 05:46:18 kernel:  entry_SYSCALL_64_after_hwframe+0x76/0x7e
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: The buggy address belongs to the object at ffff8882137cb680
                               which belongs to the cache net_namespace of size 6720
Dec 03 05:46:18 kernel: The buggy address is located 5168 bytes inside of
                               freed 6720-byte region [ffff8882137cb680, ffff8882137cd0c0)
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: The buggy address belongs to the physical page:
Dec 03 05:46:18 kernel: page: refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x2137c8
Dec 03 05:46:18 kernel: head: order:3 mapcount:0 entire_mapcount:0 nr_pages_mapped:0 pincount:0
Dec 03 05:46:18 kernel: memcg:ffff88812794d901
Dec 03 05:46:18 kernel: flags: 0x17ffffc0000040(head|node=0|zone=2|lastcpupid=0x1fffff)
Dec 03 05:46:18 kernel: page_type: f5(slab)
Dec 03 05:46:18 kernel: raw: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
Dec 03 05:46:18 kernel: raw: 0000000000000000 0000000080040004 00000001f5000000 ffff88812794d901
Dec 03 05:46:18 kernel: head: 0017ffffc0000040 ffff888100053980 dead000000000122 0000000000000000
Dec 03 05:46:18 kernel: head: 0000000000000000 0000000080040004 00000001f5000000 ffff88812794d901
Dec 03 05:46:18 kernel: head: 0017ffffc0000003 ffffea00084df201 ffffffffffffffff 0000000000000000
Dec 03 05:46:18 kernel: head: 0000000000000008 0000000000000000 00000000ffffffff 0000000000000000
Dec 03 05:46:18 kernel: page dumped because: kasan: bad access detected
Dec 03 05:46:18 kernel: 
Dec 03 05:46:18 kernel: Memory state around the buggy address:
Dec 03 05:46:18 kernel:  ffff8882137cc980: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
Dec 03 05:46:18 kernel:  ffff8882137cca00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
Dec 03 05:46:18 kernel: >ffff8882137cca80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
Dec 03 05:46:18 kernel:                                      ^
Dec 03 05:46:18 kernel:  ffff8882137ccb00: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
Dec 03 05:46:18 kernel:  ffff8882137ccb80: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
Dec 03 05:46:18 kernel: ==================================================================

This one seems to be reproducible fairly consistently.

Best regards, Ilya Maximets.

Powered by blists - more mailing lists