| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CANn89iLbC3qkeptG9xv1nZyWHUTdtXBf4w3LGaisRGc7xj4pMw@mail.gmail.com> Date: Thu, 2 Jan 2025 12:46:52 +0100 From: Eric Dumazet <edumazet@...gle.com> To: Haifeng Xu <haifeng.xu@...pee.com> Cc: Tony Nguyen <anthony.l.nguyen@...el.com>, Przemek Kitszel <przemyslaw.kitszel@...el.com>, "David S. Miller" <davem@...emloft.net>, Jakub Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>, linux-kernel@...r.kernel.org, netdev@...r.kernel.org, intel-wired-lan@...ts.osuosl.org Subject: Re: [Question] ixgbe：Mechanism of RSS On Thu, Jan 2, 2025 at 12:23 PM Haifeng Xu <haifeng.xu@...pee.com> wrote: > > > > On 2025/1/2 18:34, Eric Dumazet wrote: > > On Thu, Jan 2, 2025 at 9:43 AM Haifeng Xu <haifeng.xu@...pee.com> wrote: > >> > >> > >> > >> On 2025/1/2 16:13, Eric Dumazet wrote: > >>> On Thu, Jan 2, 2025 at 4:53 AM Haifeng Xu <haifeng.xu@...pee.com> wrote: > >>>> > >>>> Hi masters, > >>>> > >>>> We use the Intel Corporation 82599ES NIC in our production environment. And it has 63 rx queues, every rx queue interrupt is processed by a single cpu. > >>>> The RSS configuration can be seen as follow: > >>>> > >>>> RX flow hash indirection table for eno5 with 63 RX ring(s): > >>>> 0: 0 1 2 3 4 5 6 7 > >>>> 8: 8 9 10 11 12 13 14 15 > >>>> 16: 0 1 2 3 4 5 6 7 > >>>> 24: 8 9 10 11 12 13 14 15 > >>>> 32: 0 1 2 3 4 5 6 7 > >>>> 40: 8 9 10 11 12 13 14 15 > >>>> 48: 0 1 2 3 4 5 6 7 > >>>> 56: 8 9 10 11 12 13 14 15 > >>>> 64: 0 1 2 3 4 5 6 7 > >>>> 72: 8 9 10 11 12 13 14 15 > >>>> 80: 0 1 2 3 4 5 6 7 > >>>> 88: 8 9 10 11 12 13 14 15 > >>>> 96: 0 1 2 3 4 5 6 7 > >>>> 104: 8 9 10 11 12 13 14 15 > >>>> 112: 0 1 2 3 4 5 6 7 > >>>> 120: 8 9 10 11 12 13 14 15 > >>>> > >>>> The maximum number of RSS queues is 16. So I have some questions about this. Will other cpus except 0~15 receive the rx interrupts? > >>>> > >>>> In our production environment, cpu 16~62 also receive the rx interrupts. Was our RSS misconfigured? > >>> > >>> It really depends on which cpus are assigned to each IRQ. > >>> > >> > >> Hi Eric, > >> > >> Each irq was assigned to a single cpu, for exapmle: > >> > >> irq cpu > >> > >> 117 0 > >> 118 1 > >> > >> ...... > >> > >> 179 62 > >> > >> All cpus trigger interrupts not only cpus 0~15. > >> It seems that the result is inconsistent with the RSS hash value. > >> > >> > > > > I misread your report, I thought you had 16 receive queues. > > > > Why don't you change "ethtool -L eno5 rx 16", instead of trying to > > configure RSS manually ? > > Hi Eric, > > We want to make full use of cpu resources to receive packets. So > we enable 63 rx queues. But we found the rate of interrupt growth > on cpu 0~15 is faster than other cpus(almost twice). I don't know > whether it is related to RSS configuration. We didn't make any changes > on the RSS configration after the server is up. > > > > FYI, on another server, we use Mellanox Technologies MT27800 NIC. > The rate of interrupt growth on cpu 0~63 seems have little gap. > > It's RSS configration can be seen as follow: > > RX flow hash indirection table for ens2f0np0 with 63 RX ring(s): > 0: 0 1 2 3 4 5 6 7 > 8: 8 9 10 11 12 13 14 15 > 16: 16 17 18 19 20 21 22 23 > 24: 24 25 26 27 28 29 30 31 > 32: 32 33 34 35 36 37 38 39 > 40: 40 41 42 43 44 45 46 47 > 48: 48 49 50 51 52 53 54 55 > 56: 56 57 58 59 60 61 62 0 > 64: 1 2 3 4 5 6 7 8 > 72: 9 10 11 12 13 14 15 16 > 80: 17 18 19 20 21 22 23 24 > 88: 25 26 27 28 29 30 31 32 > 96: 33 34 35 36 37 38 39 40 > 104: 41 42 43 44 45 46 47 48 > 112: 49 50 51 52 53 54 55 56 > 120: 57 58 59 60 61 62 0 1 > 128: 2 3 4 5 6 7 8 9 > 136: 10 11 12 13 14 15 16 17 > 144: 18 19 20 21 22 23 24 25 > 152: 26 27 28 29 30 31 32 33 > 160: 34 35 36 37 38 39 40 41 > 168: 42 43 44 45 46 47 48 49 > 176: 50 51 52 53 54 55 56 57 > 184: 58 59 60 61 62 0 1 2 > 192: 3 4 5 6 7 8 9 10 > 200: 11 12 13 14 15 16 17 18 > 208: 19 20 21 22 23 24 25 26 > 216: 27 28 29 30 31 32 33 34 > 224: 35 36 37 38 39 40 41 42 > 232: 43 44 45 46 47 48 49 50 > 240: 51 52 53 54 55 56 57 58 > 248: 59 60 61 62 0 1 2 3 > > > I am confused that why ixgbe NIC can dispatch the packets > to the rx queues that not specified in RSS configuration. Perhaps make sure to change RX flow hash indirection table on the Intel NIC then... Maybe you changed the default configuration. ethtool -X eno5 equal 64 Or ethtool -X eno5 default
Powered by blists - more mailing lists