linux-kernel - Re: [Question] ixgbe：Mechanism of RSS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <0d98fed8-38e3-4118-82c9-26cefeb5ee7a@shopee.com>
Date: Fri, 3 Jan 2025 11:05:36 +0800
From: Haifeng Xu <haifeng.xu@...pee.com>
To: Edward Cree <ecree.xilinx@...il.com>, Eric Dumazet <edumazet@...gle.com>
Cc: Tony Nguyen <anthony.l.nguyen@...el.com>,
 Przemek Kitszel <przemyslaw.kitszel@...el.com>,
 "David S. Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>,
 Paolo Abeni <pabeni@...hat.com>, linux-kernel@...r.kernel.org,
 netdev@...r.kernel.org, intel-wired-lan@...ts.osuosl.org
Subject: Re: [Question] ixgbe：Mechanism of RSS



On 2025/1/3 00:01, Edward Cree wrote:
> On 02/01/2025 11:23, Haifeng Xu wrote:
>> We want to make full use of cpu resources to receive packets. So
>> we enable 63 rx queues. But we found the rate of interrupt growth
>> on cpu 0~15 is faster than other cpus(almost twice).
> ...
>> I am confused that why ixgbe NIC can dispatch the packets
>> to the rx queues that not specified in RSS configuration.
> 
> Hypothesis: it isn't doing so, RX is only happening on cpus (and
>  queues) 0-15, but the other CPUs are still sending traffic and
>  thus getting TX completion interrupts from their TX queues.
> `ethtool -S` output has per-queue traffic stats which should
>  confirm this.
> 

I use ethtool -S to check the rx_queus stats and here is the result.

According to the below stats, all cpus have new packets received.

cpu     t1(bytes)       t2(bytes)       delta(bytes)

0	154155550267550	154156433828875	883561325
1	148748566285840	148749509346247	943060407
2	148874911191685	148875798038140	886846455
3	152483460327704	152484251468998	791141294
4	147790981836915	147791775847804	794010889
5	146047892285722	146048778285682	885999960
6	142880516825921	142881213804363	696978442
7	152016735168735	152017707542774	972374039
8	146019936404393	146020739070311	802665918
9	147448522715540	147449258018186	735302646
10	145865736299432	145866601503106	865203674
11	149548527982122	149549289026453	761044331
12	146848384328236	146849303547769	919219533
13	152942139118542	152942769029253	629910711
14	150884661854828	150885556866976	895012148
15	149222733506734	149223510491115	776984381
16	34150226069524	34150375855113	149785589
17	34115700500819	34115914271025	213770206
18	33906215129998	33906448044501	232914503
19	33983812095357	33983986258546	174163189
20	34156349675011	34156565159083	215484072
21	33574293379024	33574490695725	197316701
22	33438129453422	33438297911151	168457729
23	32967454521585	32967612494711	157973126
24	33507443427266	33507604828468	161401202
25	33413275870121	33413433901940	158031819
26	33852322542796	33852527061150	204518354
27	33131162685385	33131330621474	167936089
28	33407661780251	33407823112381	161332130
29	34256799173845	34256944837757	145663912
30	33814458585183	33814623673528	165088345
31	33848638714862	33848775218038	136503176
32	18683932398308	18684069540891	137142583
33	19454524281229	19454647908293	123627064
34	19717744365436	19717900618222	156252786
35	20295086765202	20295245869666	159104464
36	20501853066588	20502000738936	147672348
37	20954631043374	20954797204375	166161001
38	21102911073326	21103062510369	151437043
39	21376404644179	21376515307288	110663109
40	20935812784743	20935983891491	171106748
41	20721278456831	20721435955715	157498884
42	21268291801465	21268425244578	133443113
43	21661413672829	21661629019091	215346262
44	21696437732484	21696568800049	131067565
45	21027869000890	21028020401214	151400324
46	21707137252644	21707293761990	156509346
47	20655623913790	20655740452889	116539099
48	32692002128477	32692138244468	136115991
49	33548445851486	33548569927672	124076186
50	33197264968787	33197448645817	183677030
51	33379544010500	33379746565576	202555076
52	33503579011721	33503722596159	143584438
53	33145734550468	33145892305819	157755351
54	33422692741858	33422844156764	151414906
55	32750945531107	32751131302251	185771144
56	33404955373530	33405157766253	202392723
57	33701185654471	33701313174725	127520254
58	33014531699810	33014700058409	168358599
59	32948906758429	32949151147605	244389176
60	33470813725985	33470993164755	179438770
61	33803771479735	33803971758441	200278706
62	33509751180818	33509926649969	175469151

Thanks!

> (But Eric is right that if you _want_ RX to use every CPU you
>  should just change the indirection table.)