linux-kernel - Re: [PATCH] cifs: Fix problem with encrypted RDMA data read

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <3609b064-175c-fc18-cd1a-e177d0349c58@samba.org>
Date:   Wed, 16 Nov 2022 01:57:09 +0100
From:   Stefan Metzmacher <metze@...ba.org>
To:     David Howells <dhowells@...hat.com>, smfrench@...il.com,
        tom@...pey.com
Cc:     Long Li <longli@...rosoft.com>,
        Namjae Jeon <linkinjeon@...nel.org>,
        linux-cifs@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] cifs: Fix problem with encrypted RDMA data read

Hi David,

see below...

> When the cifs client is talking to the ksmbd server by RDMA and the ksmbd
> server has "smb3 encryption = yes" in its config file, the normal PDU
> stream is encrypted, but the directly-delivered data isn't in the stream
> (and isn't encrypted), but is rather delivered by DDP/RDMA packets (at
> least with IWarp).
> 
> Currently, the direct delivery fails with:
> 
>     buf can not contain only a part of read data
>     WARNING: CPU: 0 PID: 4619 at fs/cifs/smb2ops.c:4731 handle_read_data+0x393/0x405
>     ...
>     RIP: 0010:handle_read_data+0x393/0x405
>     ...
>      smb3_handle_read_data+0x30/0x37
>      receive_encrypted_standard+0x141/0x224
>      cifs_demultiplex_thread+0x21a/0x63b
>      kthread+0xe7/0xef
>      ret_from_fork+0x22/0x30
> 
> The problem apparently stemming from the fact that it's trying to manage
> the decryption, but the data isn't in the smallbuf, the bigbuf or the page
> array).
> 
> This can be fixed simply by inserting an extra case into handle_read_data()
> that checks to see if use_rdma_mr is true, and if it is, just setting
> rdata->got_bytes to the length of data delivered and allowing normal
> continuation.
> 
> This can be seen in an IWarp packet trace.  With the upstream code, it does
> a DDP/RDMA packet, which produces the warning above and then retries,
> retrieving the data inline, spread across several SMBDirect messages that
> get glued together into a single PDU.  With the patch applied, only the
> DDP/RDMA packet is seen.
> 
> Note that this doesn't happen if the server isn't told to encrypt stuff and
> it does also happen with softRoCE.
> 
> Signed-off-by: David Howells <dhowells@...hat.com>
> cc: Steve French <smfrench@...il.com>
> cc: Tom Talpey <tom@...pey.com>
> cc: Long Li <longli@...rosoft.com>
> cc: Namjae Jeon <linkinjeon@...nel.org>
> cc: Stefan Metzmacher <metze@...ba.org>
> cc: linux-cifs@...r.kernel.org
> ---
> 
>   fs/cifs/smb2ops.c |    3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/fs/cifs/smb2ops.c b/fs/cifs/smb2ops.c
> index 880cd494afea..8d459f60f27b 100644
> --- a/fs/cifs/smb2ops.c
> +++ b/fs/cifs/smb2ops.c
> @@ -4726,6 +4726,9 @@ handle_read_data(struct TCP_Server_Info *server, struct mid_q_entry *mid,
>   		iov.iov_base = buf + data_offset;
>   		iov.iov_len = data_len;
>   		iov_iter_kvec(&iter, WRITE, &iov, 1, data_len);
> +	} else if (use_rdma_mr) {
> +		/* The data was delivered directly by RDMA. */
> +		rdata->got_bytes = data_len;
>   	} else {
>   		/* read response payload cannot be in both buf and pages */
>   		WARN_ONCE(1, "buf can not contain only a part of read data");

I'm not sure I understand why this would fix anything when encryption is enabled.

Is the payload still be offloaded as plaintext? Otherwise we wouldn't have use_rdma_mr...
So this rather looks like a fix for the non encrypted case.

Before smbd_register_mr() is called we typically have a check like this:

       if (server->rdma && !server->sign && wdata->bytes >=
               server->smbd_conn->rdma_readwrite_threshold) {

I'm wondering if server->sign is true for the encryption case, otherwise
we would have to add a !encrypt check in addition as we should never use
RDMA offload for encrypted connections.

Latest Windows servers allow encrypted/signed offload, but that needs to be
negotiated via MS-SMB2 2.2.3.1.6 SMB2_RDMA_TRANSFORM_CAPABILITIES, see
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/52b74a74-9838-4f51-b2b0-efeb23bd79d6
And SMB2_READFLAG_RESPONSE_RDMA_TRANSFORM in MS-SMB2 2.2.20 SMB2 READ Response
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/3e3d2f2c-0e2f-41ea-ad07-fbca6ffdfd90
As well as SMB2_CHANNEL_RDMA_TRANSFORM in 2.2.21 SMB2 WRITE Request
https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-smb2/e7046961-3318-4350-be2a-a8d69bb59ce8
But none of this is implemented in Linux yet.

metze