| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAFki+LnUGiEE-7Uf-x8-TQZYZ+3Migrr=81gGLYszxaK-6A9WQ@mail.gmail.com>
Date: Thu, 8 Jul 2021 15:24:20 -0400
From: Nitesh Lal <nilal@...hat.com>
To: Nitesh Narayan Lal <nitesh@...hat.com>,
linux-kernel@...r.kernel.org, linux-scsi@...r.kernel.org,
netdev@...r.kernel.org, linux-api@...r.kernel.org,
linux-pci@...r.kernel.org, Thomas Gleixner <tglx@...utronix.de>,
Jesse Brandeburg <jesse.brandeburg@...el.com>,
Robin Murphy <robin.murphy@....com>,
Marcelo Tosatti <mtosatti@...hat.com>,
Ingo Molnar <mingo@...nel.org>, jbrandeb@...nel.org,
frederic@...nel.org, Juri Lelli <juri.lelli@...hat.com>,
Alex Belits <abelits@...vell.com>,
Bjorn Helgaas <bhelgaas@...gle.com>, rostedt@...dmis.org,
peterz@...radead.org, davem@...emloft.net,
akpm@...ux-foundation.org, sfr@...b.auug.org.au,
stephen@...workplumber.org, rppt@...ux.vnet.ibm.com,
chris.friesen@...driver.com, Marc Zyngier <maz@...nel.org>,
Neil Horman <nhorman@...driver.com>, pjwaskiewicz@...il.com,
Stefan Assmann <sassmann@...hat.com>,
Tomas Henzl <thenzl@...hat.com>, kashyap.desai@...adcom.com,
Sumit Saxena <sumit.saxena@...adcom.com>,
shivasharan.srikanteshwara@...adcom.com,
sathya.prakash@...adcom.com,
Sreekanth Reddy <sreekanth.reddy@...adcom.com>,
suganath-prabu.subramani@...adcom.com, james.smart@...adcom.com,
dick.kennedy@...adcom.com, Ken Cox <jkc@...hat.com>,
faisal.latif@...el.com, shiraz.saleem@...el.com, tariqt@...dia.com,
Alaa Hleihel <ahleihel@...hat.com>,
Kamal Heib <kheib@...hat.com>, borisp@...dia.com,
saeedm@...dia.com, benve@...co.com, govind@....com,
jassisinghbrar@...il.com, ajit.khaparde@...adcom.com,
sriharsha.basavapatna@...adcom.com, somnath.kotur@...adcom.com,
"Nikolova, Tatyana E" <tatyana.e.nikolova@...el.com>,
"Ismail, Mustafa" <mustafa.ismail@...el.com>,
Al Stone <ahs3@...hat.com>, leonro@...dia.com
Subject: Re: [PATCH v2 00/14] genirq: Cleanup the usage of irq_set_affinity_hint
On Tue, Jun 29, 2021 at 11:28 AM Nitesh Narayan Lal <nitesh@...hat.com> wrote:
>
> The drivers currently rely on irq_set_affinity_hint() to either set the
> affinity_hint that is consumed by the userspace and/or to enforce a custom
> affinity.
>
> irq_set_affinity_hint() as the name suggests is originally introduced to
> only set the affinity_hint to help the userspace in guiding the interrupts
> and not the affinity itself. However, since the commit
>
> e2e64a932556 "genirq: Set initial affinity in irq_set_affinity_hint()"
>
> irq_set_affinity_hint() also started applying the provided cpumask (if not
> NULL) as the affinity for the interrupts. The issue that this commit was
> trying to solve is to allow the drivers to enforce their affinity mask to
> distribute the interrupts across the CPUs such that they don't always end
> up on CPU0. This issue has been resolved within the irq subsystem since the
> commit
>
> a0c9259dc4e1 "irq/matrix: Spread interrupts on allocation"
>
> Hence, there is no need for the drivers to overwrite the affinity to spread
> as it is dynamically performed at the time of allocation.
>
> Also, irq_set_affinity_hint() setting affinity unconditionally introduces
> issues for the drivers that only want to set their affinity_hint and not the
> affinity itself as for these driver interrupts the default_smp_affinity_mask
> is completely ignored (for detailed investigation please refer to [1]).
>
> Unfortunately reverting the commit e2e64a932556 is not an option at this
> point for two reasons [2]:
>
> - Several drivers for a valid reason (performance) rely on this API to
> enforce their affinity mask
>
> - Until very recently this was the only exported interface that was
> available
>
> To clear this out Thomas has come up with the following interfaces:
>
> - irq_set_affinity(): only sets affinity of an IRQ [3]
> - irq_update_affinity_hint(): Only sets the hint [4]
> - irq_set_affinity_and_hint(): Sets both affinity and the hint mask [4]
>
> The first API is already merged in the linux-next tree and the patch
> that introduces the other two interfaces are included with this patch-set.
>
> To move to the stage where we can safely get rid of the
> irq_set_affinity_hint(), which has been marked deprecated, we have to
> move all its consumers to these new interfaces. In this patch-set, I have
> done that for a few drivers and will hopefully try to move the remaining of
> them in the coming days.
>
> Testing
> -------
> In terms of testing, I have performed some basic testing on x86 to verify
> things such as the interrupts are evenly spread on all CPUs, hint mask is
> correctly set etc. for the drivers - i40e, iavf, mlx5, mlx4, ixgbe, i40iw
> and enic on top of:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
>
> So more testing is probably required for these and the drivers that I didn't
> test and any help will be much appreciated.
>
>
> Notes
> -----
> - I was told that i40iw driver is going to be replaced by irdma, however,
> the new driver didn't land in Linus's tree yet. Once it does I will send
> a follow up patch for that as well.
>
> - For the mpt3sas driver I decided to go with the usage of
> irq_set_affinity_and_hint over irq_set_affinity based on my little
> analysis of it and the megaraid driver. However, if we are sure that it
> is not required then I can replace it with just irq_set_affinity as one
> of its comment suggests.
>
>
> Change from v1 [5]
> ------------------
> - Fixed compilation error by adding the new interface definitions for cases
> where CONFIG_SMP is not defined
>
> - Fixed function usage in megaraid_sas and removed unnecessary variable
> (Robin Murphy)
>
> - Removed unwanted #if/endif from mlx4 (Leon Romanovsky)
>
> - Other indentation related fixes
>
>
> [1] https://lore.kernel.org/lkml/1a044a14-0884-eedb-5d30-28b4bec24b23@redhat.com/
> [2] https://lore.kernel.org/linux-pci/d1d5e797-49ee-4968-88c6-c07119343492@arm.com/
> [3] https://lore.kernel.org/linux-arm-kernel/20210518091725.046774792@linutronix.de/
> [4] https://lore.kernel.org/patchwork/patch/1434326/
> [5] https://lore.kernel.org/linux-scsi/20210617182242.8637-1-nitesh@redhat.com/
>
>
> Nitesh Narayan Lal (13):
> iavf: Use irq_update_affinity_hint
> i40e: Use irq_update_affinity_hint
> scsi: megaraid_sas: Use irq_set_affinity_and_hint
> scsi: mpt3sas: Use irq_set_affinity_and_hint
> RDMA/i40iw: Use irq_update_affinity_hint
> enic: Use irq_update_affinity_hint
> be2net: Use irq_update_affinity_hint
> ixgbe: Use irq_update_affinity_hint
> mailbox: Use irq_update_affinity_hint
> scsi: lpfc: Use irq_set_affinity
> hinic: Use irq_set_affinity_and_hint
> net/mlx5: Use irq_update_affinity_hint
> net/mlx4: Use irq_update_affinity_hint
>
> Thomas Gleixner (1):
> genirq: Provide new interfaces for affinity hints
>
> drivers/infiniband/hw/i40iw/i40iw_main.c | 4 +-
> drivers/mailbox/bcm-flexrm-mailbox.c | 4 +-
> drivers/net/ethernet/cisco/enic/enic_main.c | 8 +--
> drivers/net/ethernet/emulex/benet/be_main.c | 4 +-
> drivers/net/ethernet/huawei/hinic/hinic_rx.c | 4 +-
> drivers/net/ethernet/intel/i40e/i40e_main.c | 8 +--
> drivers/net/ethernet/intel/iavf/iavf_main.c | 8 +--
> drivers/net/ethernet/intel/ixgbe/ixgbe_main.c | 10 ++--
> drivers/net/ethernet/mellanox/mlx4/eq.c | 8 ++-
> .../net/ethernet/mellanox/mlx5/core/pci_irq.c | 6 +--
> drivers/scsi/lpfc/lpfc_init.c | 4 +-
> drivers/scsi/megaraid/megaraid_sas_base.c | 27 +++++-----
> drivers/scsi/mpt3sas/mpt3sas_base.c | 21 ++++----
> include/linux/interrupt.h | 53 ++++++++++++++++++-
> kernel/irq/manage.c | 8 +--
> 15 files changed, 113 insertions(+), 64 deletions(-)
>
> --
>
>
Gentle ping.
Any comments or suggestions on any of the patches included in this series?
--
Thanks
Nitesh
Powered by blists - more mailing lists