lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  PHC 
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Sat, 29 Jan 2022 11:43:22 +0800
From:   Guangguan Wang <>
To:     Stefan Raspl <>,
Subject: Re: [RFC PATCH net-next] net/smc: Introduce receive queue flow
 control support

On 2022/1/25 17:42, Stefan Raspl wrote:
> That's some truly substantial improvements!
> But we need to be careful with protocol-level changes: There are other operating systems like z/OS and AIX which have compatible implementations of SMC, too. Changes like a reduction of connections per link group or usage of reserved fields would need to be coordinated, and likely would have unwanted side-effects even when used with older Linux kernel versions.
> Changing the protocol is "expensive" insofar as it requires time to thoroughly discuss the changes, perform compatibility tests, and so on.
> So I would like to urge you to investigate alternative ways that do not require protocol-level changes to address this scenario, e.g. by modifying the number of completion queue elements, to see if this could yield similar results.
> Thx!

Yes, there are alternative ways, as RNR caused by the missmatch of send rate and receive rate, which means sending too fast
or receiving too slow. What I have done in this patch is to backpressure the sending side when sending too fast.

Another solution is to process and refill the receive queue as quickly as posibble, which requires no protocol-level change. 
The fllowing modifications are needed:
- Enqueue cdc msgs to backlog queues instead of processing in rx tasklet. llc msgs remain unchanged.
- A mempool is needed as cdc msgs are processed asynchronously. Allocate new receive buffers from mempool when refill receive queue.
- Schedule backlog queues to other cpus, which are calculated by 4-tuple or 5-tuple hash of the connections, to process the cdc msgs,
  in order to reduce the usage of the cpu where rx tasklet runs on.

the pseudocode shows below:
    if cdc_msgs
        enqueue to backlog;
	maybe smp_call_function_single_async is needed to wakeup the corresponding cpu to process backlog;
        allocate new buffer and modify the sge in rq_wr;
        process remains unchanged;

    post_recv rq_wr;
end rx_tasklet

smp_backlog_process in corresponding cpu, called by smp_call_function_single_async
    for connections hashed to this cpu
        for cdc_msgs in backlog
            process cdc msgs;
        end cdc_msgs
    end connections
end smp_backlog_process

I‘d like to hear your suggestions of this solution.
Thank you.

Powered by blists - more mailing lists