[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <eed8b91e-0236-edc8-f744-e30adfff229f@linux.alibaba.com>
Date: Wed, 8 Feb 2023 10:58:25 +0800
From: "D. Wythe" <alibuda@...ux.alibaba.com>
To: Wenjia Zhang <wenjia@...ux.ibm.com>, kgraul@...ux.ibm.com,
jaka@...ux.ibm.com
Cc: kuba@...nel.org, davem@...emloft.net, netdev@...r.kernel.org,
linux-s390@...r.kernel.org, linux-rdma@...r.kernel.org
Subject: Re: [net-next 1/2] net/smc: allow confirm/delete rkey response
deliver multiplex
On 2/8/23 7:15 AM, Wenjia Zhang wrote:
>
>
> On 07.02.23 08:36, D. Wythe wrote:
>> From: "D. Wythe" <alibuda@...ux.alibaba.com>
>>
>> We know that all flows except confirm_rkey and delete_rkey are exclusive,
>> confirm/delete rkey flows can run concurrently (local and remote).
>>
>> Although the protocol allows, all flows are actually mutually exclusive
>> in implementation, dues to waiting for LLC messages is in serial.
>>
>> This aggravates the time for establishing or destroying a SMC-R
>> connections, connections have to be queued in smc_llc_wait.
>>
>> We use rtokens or rkey to correlate a confirm/delete rkey message
>> with its response.
>>
>> Before sending a message, we put context with rtokens or rkey into
>> wait queue. When a response message received, we wakeup the context
>> which with the same rtokens or rkey against the response message.
>>
>> Signed-off-by: D. Wythe <alibuda@...ux.alibaba.com>
>> ---
>> net/smc/smc_llc.c | 174 +++++++++++++++++++++++++++++++++++++++++-------------
>> net/smc/smc_wr.c | 10 ----
>> net/smc/smc_wr.h | 10 ++++
>> 3 files changed, 143 insertions(+), 51 deletions(-)
>>
>
> [...]
>
>> +static int smc_llc_rkey_response_wake_function(struct wait_queue_entry *wq_entry,
>> + unsigned int mode, int sync, void *key)
>> +{
>> + struct smc_llc_qentry *except, *incoming;
>> + u8 except_llc_type;
>> +
>> + /* not a rkey response */
>> + if (!key)
>> + return 0;
>> +
>> + except = wq_entry->private;
>> + incoming = key;
>> +
>> + except_llc_type = except->msg.raw.hdr.common.llc_type;
>> +
>> + /* except LLC MSG TYPE mismatch */
>> + if (except_llc_type != incoming->msg.raw.hdr.common.llc_type)
>> + return 0;
>> +
>> + switch (except_llc_type) {
>> + case SMC_LLC_CONFIRM_RKEY:
>> + if (memcmp(except->msg.confirm_rkey.rtoken, incoming->msg.confirm_rkey.rtoken,
>> + sizeof(struct smc_rmb_rtoken) *
>> + except->msg.confirm_rkey.rtoken[0].num_rkeys))
>> + return 0;
>> + break;
>> + case SMC_LLC_DELETE_RKEY:
>> + if (memcmp(except->msg.delete_rkey.rkey, incoming->msg.delete_rkey.rkey,
>> + sizeof(__be32) * except->msg.delete_rkey.num_rkeys))
>> + return 0;
>> + break;
>> + default:
>> + pr_warn("smc: invalid except llc msg %d.\n", except_llc_type);
>> + return 0;
>> + }
>> +
>> + /* match, save hdr */
>> + memcpy(&except->msg.raw.hdr, &incoming->msg.raw.hdr, sizeof(except->msg.raw.hdr));
>> +
>> + wq_entry->private = except->private;
>> + return woken_wake_function(wq_entry, mode, sync, NULL);
>> +}
>> +
>
> s/except/expect/ ?
> Just kind of confusing
>
> [...]
Hi, Wenjia
Except is what I want to express.
It means that only the confirm and delete rkey can be processed in parallel. :-)
Thanks
D. Wythe
Powered by blists - more mailing lists