netdev - RE: __assign_irq_vector (x86) and irq vectors exhaust

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <063D6719AE5E284EB5DD2968C1650D6D1CB37AD1@AcuExch.aculab.com>
Date:	Mon, 18 May 2015 13:40:52 +0000
From:	David Laight <David.Laight@...LAB.COM>
To:	"'p.kosyh@...il.com'" <p.kosyh@...il.com>,
	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: __assign_irq_vector (x86) and irq vectors exhaust

From: p.kosyh@...il.com
> Sent: 18 May 2015 13:58
> Playing with 10Gb network adapters and /proc/irq/<nr>/smp_affinity
> we found that sometimes we can not move interrupt on selected cpu.
> 
> After digging source code we found, that
> arch/x86/kernel/apic/vector.c: __assign_irq_vector (4.0 kernel)
> allocates vectors in not optimal way.
> 
> For example, we have a 32 cpu system with lot of 10Gb cards (each of
> them has 32 msi-x irqs). Even if card is not used, it allocates an irq
> vector after probing (pci_enable_msix()). We have about ~200 vectors limit
> per cpu (on x86), and __assign_irq_vector allocates them filling cpus one
> by one (see at cpumask_first_and()):
...

It might help if the kernel APIs allowed a driver to request additional
MSI-X interrupts after probe time.

If a device supports 32 interrupts the driver can say that it only
needs (say) interrupts 0, 1 and 16 (and only these MSIX table slots
get filled with interrupt 'info') - but can't later allocate the
MSIX info for other interrupts.

I can't see anything in the MSIX spec that stops things working
that way.

	David