| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <6bff9779-468e-4430-861c-720f63708dbb@samba.org>
Date: Thu, 27 Nov 2025 11:57:27 +0100
From: Stefan Metzmacher <metze@...ba.org>
To: Zhu Yanjun <yanjun.zhu@...ux.dev>, linux-rdma@...r.kernel.org
Cc: Zhu Yanjun <zyjzyj2000@...il.com>, Jason Gunthorpe <jgg@...pe.ca>,
Leon Romanovsky <leon@...nel.org>, netdev@...r.kernel.org,
linux-cifs@...r.kernel.org
Subject: Re: [PATCH v2] RDMA/rxe: reclassify sockets in order to avoid false
positives from lockdep
Am 27.11.25 um 05:43 schrieb Zhu Yanjun:
>
> 在 2025/11/26 7:07, Stefan Metzmacher 写道:
>> While developing IPPROTO_SMBDIRECT support for the code
>> under fs/smb/common/smbdirect [1], I noticed false positives like this:
>>
>> [+0,003927] ============================================
>> [+0,000532] WARNING: possible recursive locking detected
>> [+0,000611] 6.18.0-rc5-metze-kasan-lockdep.02+ #1 Tainted: G OE
>> [+0,000835] --------------------------------------------
>> [+0,000729] ksmbd:r5445/3609 is trying to acquire lock:
>> [+0,000709] ffff88800b9570f8 (k-sk_lock-AF_INET){+.+.}-{0:0},
>> at: inet_shutdown+0x52/0x360
>> [+0,000831]
>> but task is already holding lock:
>> [+0,000684] ffff88800654af78 (k-sk_lock-AF_INET){+.+.}-{0:0},
>> at: smbdirect_sk_close+0x122/0x790 [smbdirect]
>> [+0,000928]
>> other info that might help us debug this:
>> [+0,005552] Possible unsafe locking scenario:
>>
>> [+0,000723] CPU0
>> [+0,000359] ----
>> [+0,000377] lock(k-sk_lock-AF_INET);
>> [+0,000478] lock(k-sk_lock-AF_INET);
>> [+0,000498]
>> *** DEADLOCK ***
>>
>> [+0,001012] May be due to missing lock nesting notation
>>
>> [+0,000831] 3 locks held by ksmbd:r5445/3609:
>> [+0,000484] #0: ffff88800654af78 (k-sk_lock-AF_INET){+.+.}-{0:0},
>> at: smbdirect_sk_close+0x122/0x790 [smbdirect]
>> [+0,001000] #1: ffff888020a40458 (&id_priv->handler_mutex){+.+.}-{4:4},
>> at: rdma_lock_handler+0x17/0x30 [rdma_cm]
>> [+0,000982] #2: ffff888020a40350 (&id_priv->qp_mutex){+.+.}-{4:4},
>> at: rdma_destroy_qp+0x5d/0x1f0 [rdma_cm]
>> [+0,000934]
>> stack backtrace:
>> [+0,000589] CPU: 0 UID: 0 PID: 3609 Comm: ksmbd:r5445 Kdump: loaded
>> Tainted: G OE
>> 6.18.0-rc5-metze-kasan-lockdep.02+ #1 PREEMPT(voluntary)
>> [+0,000023] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
>> [+0,000004] Hardware name: innotek GmbH VirtualBox/VirtualBox,
>> BIOS VirtualBox 12/01/2006
>> ...
>> [+0,000010] print_deadlock_bug+0x245/0x330
>> [+0,000014] validate_chain+0x32a/0x590
>> [+0,000012] __lock_acquire+0x535/0xc30
>> [+0,000013] lock_acquire.part.0+0xb3/0x240
>> [+0,000017] ? inet_shutdown+0x52/0x360
>> [+0,000013] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000007] ? mark_held_locks+0x46/0x90
>> [+0,000012] lock_acquire+0x60/0x140
>> [+0,000006] ? inet_shutdown+0x52/0x360
>> [+0,000028] lock_sock_nested+0x3b/0xf0
>> [+0,000009] ? inet_shutdown+0x52/0x360
>> [+0,000008] inet_shutdown+0x52/0x360
>> [+0,000010] kernel_sock_shutdown+0x5b/0x90
>> [+0,000011] rxe_qp_do_cleanup+0x4ef/0x810 [rdma_rxe]
>> [+0,000043] ? __pfx_rxe_qp_do_cleanup+0x10/0x10 [rdma_rxe]
>> [+0,000030] execute_in_process_context+0x2b/0x170
>> [+0,000013] rxe_qp_cleanup+0x1c/0x30 [rdma_rxe]
>> [+0,000021] __rxe_cleanup+0x1cf/0x2e0 [rdma_rxe]
>> [+0,000036] ? __pfx___rxe_cleanup+0x10/0x10 [rdma_rxe]
>> [+0,000020] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000006] ? __kasan_check_read+0x11/0x20
>> [+0,000012] rxe_destroy_qp+0xe1/0x230 [rdma_rxe]
>> [+0,000035] ib_destroy_qp_user+0x217/0x450 [ib_core]
>> [+0,000074] rdma_destroy_qp+0x83/0x1f0 [rdma_cm]
>> [+0,000034] smbdirect_connection_destroy_qp+0x98/0x2e0 [smbdirect]
>> [+0,000017] ? __pfx_smb_direct_logging_needed+0x10/0x10 [ksmbd]
>> [+0,000044] smbdirect_connection_destroy+0x698/0xed0 [smbdirect]
>> [+0,000023] ? __pfx_smbdirect_connection_destroy+0x10/0x10 [smbdirect]
>> [+0,000033] ? __pfx_smb_direct_logging_needed+0x10/0x10 [ksmbd]
>> [+0,000031] smbdirect_connection_destroy_sync+0x42b/0x9f0 [smbdirect]
>> [+0,000029] ? mark_held_locks+0x46/0x90
>> [+0,000012] ? __pfx_smbdirect_connection_destroy_sync+0x10/0x10 [smbdirect]
>> [+0,000019] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000007] ? trace_hardirqs_on+0x64/0x70
>> [+0,000029] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000010] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000006] ? __smbdirect_connection_schedule_disconnect+0x339/0x4b0
>> [+0,000021] smbdirect_sk_destroy+0xb0/0x680 [smbdirect]
>> [+0,000024] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000006] ? trace_hardirqs_on+0x64/0x70
>> [+0,000006] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000005] ? __local_bh_enable_ip+0xba/0x150
>> [+0,000011] sk_common_release+0x66/0x340
>> [+0,000010] smbdirect_sk_close+0x12a/0x790 [smbdirect]
>> [+0,000023] ? ip_mc_drop_socket+0x1e/0x240
>> [+0,000013] inet_release+0x10a/0x240
>> [+0,000011] smbdirect_sock_release+0x502/0xe80 [smbdirect]
>> [+0,000015] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000024] sock_release+0x91/0x1c0
>> [+0,000010] smb_direct_free_transport+0x31/0x50 [ksmbd]
>> [+0,000025] ksmbd_conn_free+0x1d0/0x240 [ksmbd]
>> [+0,000040] smb_direct_disconnect+0xb2/0x120 [ksmbd]
>> [+0,000023] ? srso_alias_return_thunk+0x5/0xfbef5
>> [+0,000018] ksmbd_conn_handler_loop+0x94e/0xf10 [ksmbd]
>> ...
>>
>> I'll also add reclassify to the smbdirect socket code [1],
>> but I think it's better to have it in both direction
>> (below and above the RDMA layer).
>>
>> [1]
>> https://git.samba.org/?p=metze/linux/wip.git;a=shortlog;h=refs/heads/master-ipproto-smbdirect
>>
>> Cc: Zhu Yanjun <zyjzyj2000@...il.com>
>> Cc: Jason Gunthorpe <jgg@...pe.ca>
>> Cc: Leon Romanovsky <leon@...nel.org>
>> Cc: linux-rdma@...r.kernel.org
>> Cc: netdev@...r.kernel.org
>> Cc: linux-cifs@...r.kernel.org
>> Signed-off-by: Stefan Metzmacher <metze@...ba.org>
>>
>> ---
>>
>> v2: - use CONFIG_DEBUG_LOCK_ALLOC (Bernard on siw patch)
>> - add a comment (Bernard on siw patch)
>> - AF_INET vs. AF_INET6 (Bernard on siw patch)
>> ---
>> drivers/infiniband/sw/rxe/rxe_net.c | 49 +++++++++++++++++++++++++++++
>> drivers/infiniband/sw/rxe/rxe_qp.c | 49 +++++++++++++++++++++++++++++
>> 2 files changed, 98 insertions(+)
>>
>> diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c
>> index ac0183a2ff7a..9fee969f5c6f 100644
>> --- a/drivers/infiniband/sw/rxe/rxe_net.c
>> +++ b/drivers/infiniband/sw/rxe/rxe_net.c
>> @@ -18,6 +18,54 @@
>> #include "rxe_net.h"
>> #include "rxe_loc.h"
>> +#ifdef CONFIG_DEBUG_LOCK_ALLOC
>> +/*
>> + * lockdep can detect false positive circular dependencies
>> + * when there are user-space socket API users or in kernel
>> + * users switching between a tcp and rdma transport.
>> + * Maybe also switching between siw and rxe may cause
>> + * problems as per default sockets are only classified
>> + * by family and not by ip protocol. And there might
>> + * be different locks used between the application
>> + * and the low level sockets.
>> + *
>> + * Problems were seen with ksmbd.ko and cifs.ko,
>> + * switching transports, use git blame to find
>> + * more details.
>> + */
>> +static struct lock_class_key rxe_recv_sk_key[2];
>> +static struct lock_class_key rxe_recv_slock_key[2];
>> +#endif /* CONFIG_DEBUG_LOCK_ALLOC */
>> +
>> +static inline void rxe_reclassify_recv_socket(struct socket *sock)
>> +{
>> +#ifdef CONFIG_DEBUG_LOCK_ALLOC
>> + struct sock *sk = sock->sk;
>> +
>> + if (WARN_ON_ONCE(!sock_allow_reclassification(sk)))
>> + return;
>> +
>> + switch (sk->sk_family) {
>> + case AF_INET:
>> + sock_lock_init_class_and_name(sk,
>> + "slock-AF_INET-RDMA-RXE-RECV",
>> + &rxe_recv_slock_key[0],
>> + "sk_lock-AF_INET-RDMA-RXE-RECV",
>> + &rxe_recv_sk_key[0]);
>> + break;
>> + case AF_INET6:
>> + sock_lock_init_class_and_name(sk,
>> + "slock-AF_INET6-RDMA-RXE-RECV",
>> + &rxe_recv_slock_key[1],
>> + "sk_lock-AF_INET6-RDMA-RXE-RECV",
>> + &rxe_recv_sk_key[1]);
>> + break;
>> + default:
>> + WARN_ON_ONCE(1);
>> + }
>> +#endif /* CONFIG_DEBUG_LOCK_ALLOC */
>> +}
>> +
>> static struct rxe_recv_sockets recv_sockets;
>
>
> Thanks. The global variable recv_sockets should be placed at the top of the newly added source file.
> If I recall correctly, the global variable declaration needs to be at the beginning of the file and after the header files.
>
> I also read the commit of siw. This commit is similar with the one of siw.
>
> And after applying this commit, RXE still can work well. I am fine with this.
>
> Reviewed-by: Zhu Yanjun <yanjun.zhu@...ux.dev>
I just sent v3 with recv_sockets moved to the top again.
Thanks!
metze
Powered by blists - more mailing lists