linux-kernel - Re: [Question] ixgbe：Mechanism of RSS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <32775382-9079-4652-9cd5-ff0aa6b5fd9e@intel.com>
Date: Tue, 7 Jan 2025 09:16:56 -0800
From: Tony Nguyen <anthony.l.nguyen@...el.com>
To: Haifeng Xu <haifeng.xu@...pee.com>, Edward Cree <ecree.xilinx@...il.com>,
	Eric Dumazet <edumazet@...gle.com>, Aleksandr Loktionov
	<aleksandr.loktionov@...el.com>, "Kwapulinski, Piotr"
	<piotr.kwapulinski@...el.com>
CC: Przemek Kitszel <przemyslaw.kitszel@...el.com>, "David S. Miller"
	<davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, Paolo Abeni
	<pabeni@...hat.com>, <linux-kernel@...r.kernel.org>,
	<netdev@...r.kernel.org>, <intel-wired-lan@...ts.osuosl.org>
Subject: Re: [Question] ixgbe：Mechanism of RSS



On 1/2/2025 7:05 PM, Haifeng Xu wrote:
> 
> 
> On 2025/1/3 00:01, Edward Cree wrote:
>> On 02/01/2025 11:23, Haifeng Xu wrote:
>>> We want to make full use of cpu resources to receive packets. So
>>> we enable 63 rx queues. But we found the rate of interrupt growth
>>> on cpu 0~15 is faster than other cpus(almost twice).
>> ...
>>> I am confused that why ixgbe NIC can dispatch the packets
>>> to the rx queues that not specified in RSS configuration.
>>
>> Hypothesis: it isn't doing so, RX is only happening on cpus (and
>>   queues) 0-15, but the other CPUs are still sending traffic and
>>   thus getting TX completion interrupts from their TX queues.
>> `ethtool -S` output has per-queue traffic stats which should
>>   confirm this.
>>
> 
> I use ethtool -S to check the rx_queus stats and here is the result.
> 
> According to the below stats, all cpus have new packets received.

+ Alex and Piotr

What's your ntuple filter setting? If it's off, I suspect it may be the 
Flow Director ATR (Application Targeting Routing) feature which will 
utilize all queues. I believe if you turn on ntuple filters this will 
turn that feature off.

Thanks,
Tony

> 
> cpu     t1(bytes)       t2(bytes)       delta(bytes)
> 
> 0	154155550267550	154156433828875	883561325
> 1	148748566285840	148749509346247	943060407
> 2	148874911191685	148875798038140	886846455
> 3	152483460327704	152484251468998	791141294
> 4	147790981836915	147791775847804	794010889
> 5	146047892285722	146048778285682	885999960
> 6	142880516825921	142881213804363	696978442
> 7	152016735168735	152017707542774	972374039
> 8	146019936404393	146020739070311	802665918
> 9	147448522715540	147449258018186	735302646
> 10	145865736299432	145866601503106	865203674
> 11	149548527982122	149549289026453	761044331
> 12	146848384328236	146849303547769	919219533
> 13	152942139118542	152942769029253	629910711
> 14	150884661854828	150885556866976	895012148
> 15	149222733506734	149223510491115	776984381
> 16	34150226069524	34150375855113	149785589
> 17	34115700500819	34115914271025	213770206
> 18	33906215129998	33906448044501	232914503
> 19	33983812095357	33983986258546	174163189
> 20	34156349675011	34156565159083	215484072
> 21	33574293379024	33574490695725	197316701
> 22	33438129453422	33438297911151	168457729
> 23	32967454521585	32967612494711	157973126
> 24	33507443427266	33507604828468	161401202
> 25	33413275870121	33413433901940	158031819
> 26	33852322542796	33852527061150	204518354
> 27	33131162685385	33131330621474	167936089
> 28	33407661780251	33407823112381	161332130
> 29	34256799173845	34256944837757	145663912
> 30	33814458585183	33814623673528	165088345
> 31	33848638714862	33848775218038	136503176
> 32	18683932398308	18684069540891	137142583
> 33	19454524281229	19454647908293	123627064
> 34	19717744365436	19717900618222	156252786
> 35	20295086765202	20295245869666	159104464
> 36	20501853066588	20502000738936	147672348
> 37	20954631043374	20954797204375	166161001
> 38	21102911073326	21103062510369	151437043
> 39	21376404644179	21376515307288	110663109
> 40	20935812784743	20935983891491	171106748
> 41	20721278456831	20721435955715	157498884
> 42	21268291801465	21268425244578	133443113
> 43	21661413672829	21661629019091	215346262
> 44	21696437732484	21696568800049	131067565
> 45	21027869000890	21028020401214	151400324
> 46	21707137252644	21707293761990	156509346
> 47	20655623913790	20655740452889	116539099
> 48	32692002128477	32692138244468	136115991
> 49	33548445851486	33548569927672	124076186
> 50	33197264968787	33197448645817	183677030
> 51	33379544010500	33379746565576	202555076
> 52	33503579011721	33503722596159	143584438
> 53	33145734550468	33145892305819	157755351
> 54	33422692741858	33422844156764	151414906
> 55	32750945531107	32751131302251	185771144
> 56	33404955373530	33405157766253	202392723
> 57	33701185654471	33701313174725	127520254
> 58	33014531699810	33014700058409	168358599
> 59	32948906758429	32949151147605	244389176
> 60	33470813725985	33470993164755	179438770
> 61	33803771479735	33803971758441	200278706
> 62	33509751180818	33509926649969	175469151
> 
> Thanks!
> 
>> (But Eric is right that if you _want_ RX to use every CPU you
>>   should just change the indirection table.)
>