linux-kernel - Re: [PATCH 0/2] nvmet: support polling task for RDMA and TCP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <0779b376-38e3-42ef-b32a-a9cfab2749f2@grimberg.me>
Date: Sun, 30 Jun 2024 11:58:55 +0300
From: Sagi Grimberg <sagi@...mberg.me>
To: Ping Gan <jacky_gam_2001@....com>, hch@....de, kch@...dia.com,
 linux-nvme@...ts.infradead.org, linux-kernel@...r.kernel.org
Cc: ping.gan@...l.com
Subject: Re: [PATCH 0/2] nvmet: support polling task for RDMA and TCP

Hey Ping Gan,

On 26/06/2024 11:28, Ping Gan wrote:
> When running nvmf on SMP platform, current nvme target's RDMA and
> TCP use kworker to handle IO. But if there is other high workload
> in the system(eg: on kubernetes), the competition between the
> kworker and other workload is very radical. And since the kworker
> is scheduled by OS randomly, it's difficult to control OS resource
> and also tune the performance. If target support to use delicated
> polling task to handle IO, it's useful to control OS resource and
> gain good performance. So it makes sense to add polling task in
> rdma-rdma and rdma-tcp modules.

This is NOT the way to go here.

Both rdma and tcp are driven from workqueue context, which are bound 
workqueues.

So there are two ways to go here:
1. Add generic port cpuset and use that to direct traffic to the 
appropriate set of cores
(i.e. select an appropriate comp_vector for rdma and add an appropriate 
steering rule
for tcp).
2. Add options to rdma/tcp to use UNBOUND workqueues, and allow users to 
control
these UNBOUND workqueues cpumask via sysfs.

(2) will not control interrupts to steer to other workloads cpus, but 
the handlers may
run on a set of dedicated cpus.

(1) is a better solution, but harder to implement.

You also should look into nvmet-fc as well (and nvmet-loop for that matter).