[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAAhV-H6uZWgZQsVh=1-U2B4ZZZz6EPJ3gkv0mxHSNGOMPB=VwQ@mail.gmail.com>
Date: Tue, 30 May 2023 16:34:15 +0800
From: Huacai Chen <chenhuacai@...il.com>
To: Thomas Gleixner <tglx@...utronix.de>
Cc: Marc Zyngier <maz@...nel.org>,
Huacai Chen <chenhuacai@...ngson.cn>,
Bjorn Helgaas <bhelgaas@...gle.com>,
linux-kernel@...r.kernel.org, loongson-kernel@...ts.loongnix.cn,
Xuefeng Li <lixuefeng@...ngson.cn>,
Jiaxun Yang <jiaxun.yang@...goat.com>
Subject: Re: [PATCH 1/2] genirq/msi, platform-msi: Adjust return value of msi_domain_prepare_irqs()
Hi, Thomas,
On Tue, May 30, 2023 at 4:19 AM Thomas Gleixner <tglx@...utronix.de> wrote:
>
> Huacai!
>
> On Mon, May 29 2023 at 17:36, Huacai Chen wrote:
> > On Mon, May 29, 2023 at 5:27 PM Thomas Gleixner <tglx@...utronix.de> wrote:
> >> By default you allow up to 256 interrupts to be allocated, right? So to
> >> prevent vector exhaustion, the admin needs to reboot the machine and set
> >> a command line parameter to limit this, right? As that parameter is not
> >> documented the admin is going to dice a number. That's impractical and
> >> just a horrible bandaid.
> >
> > OK, I think I should update the documents in the new version.
>
> Updating documentation neither makes it more practical (it still
> requires a reboot) nor does it justify the abuse of the msi_prepare()
> callback.
>
> The only reason why this hack "works" is that there is a historical
> mechanism which tells the PCI/MSI core that the number of requested
> vectors cannot be allocated, but that there would be $N vectors
> possible. But even that return value has no guarantee.
>
> This mechanism is ill defined and really should go away.
>
> Adding yet another way to limit this via msi_prepare() is just
> proliferating this ill defined mechanism and I have zero interest in
> that.
>
> Let's take a step back and look at the larger picture:
>
> 1) A PCI/MSI irqdomain is attached to a PCI bus
>
> 2) The number of PCI devices on that PCI bus is usually known at boot
> time _before_ the first device driver is probed.
>
> That's not entirely true for PCI hotplug devices, but that's hardly
> relevant for an architecture which got designed less than 10 years
> ago and the architects decided that 256 MSI vectors are good enough
> for up to 256 CPUs. The concept of per CPU queues was already known
> at that time, no?
Does this solution depend on the per-device msi domain? Can we do that
if we use the global msi domain?
>
> So the irqdomain can tell the PCI/MSI core the maximum number of vectors
> available for a particular bus, right?
>
> The default, i.e if the irqdomain does not expose that information,
> would be "unlimited", i.e. ULONG_MAX.
OK, thanks, but how to expose? By msi_domain_info::hwsize?
>
> Now take that number and divide it by the number of devices on the bus
> and you get at least a sensible limit which does not immediately cause
> vector exhaustion.
>
> That limit might be suboptimal if there are lots of other devices on
> that bus which just require one or two vectors, but that's something
> which can be optimized via a generic command line option or even a sysfs
> mechanism.
Hmm, if we still use the command line, then we still have some similar
drawbacks.
Huacai
>
> Thanks,
>
> tglx
>
>
>
>
>
Powered by blists - more mailing lists