| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 28 Apr 2014 21:02:31 -0500
From: Suravee Suthikulanit <suravee.suthikulpanit@....com>
To: Daniel J Blueman <daniel@...ascale.com>,
Myron Stowe <myron.stowe@...il.com>,
Borislav Petkov <bp@...e.de>
CC: Myron Stowe <myron.stowe@...hat.com>,
Bjorn Helgaas <bhelgaas@...gle.com>,
linux-pci <linux-pci@...r.kernel.org>,
Aravind Gopalakrishnan <aravind.gopalakrishnan@....com>,
<kim.naru@....com>, Thomas Gleixner <tglx@...utronix.de>,
<mingo@...hat.com>, <hpa@...or.com>, x86 <x86@...nel.org>,
<sp@...ascale.com>,
"linux-acpi@...r.kernel.org" <linux-acpi@...r.kernel.org>,
LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2 5/5] PCI: Remove redundant 'quirk_amd_nb_node'
On 4/21/2014 11:53 AM, Daniel J Blueman wrote:
> Hi Myron,
>
> On 04/20/2014 09:44 PM, Myron Stowe wrote:
>> On Sun, Apr 20, 2014 at 4:54 AM, Borislav Petkov <bp@...e.de> wrote:
>>> On Fri, Apr 18, 2014 at 08:53:46PM -0600, Myron Stowe wrote:
>>>> With the amd_bus.c updates to support additional AMD processors
>>>> (11h, 12h,
>>>> 14h 15h and 16h) 'quirk_amd_nb_node' seems to be redundant. This
>>>> patch
>>>> removes it.
>>>>
>>>> Signed-off-by: Myron Stowe <myron.stowe@...hat.com>
>>>
>>> Good idea. Has this been tested, though? There are a bunch of places
>>> using dev_to_node() which this quirk corrects, especially 'local_cpus'
>>> in a bunch of sysfs nodes in the PCI hierarchy /sys/devices/pci*.
>>
>> Borislav:
>>
>> Thanks for the review, suggestions, and comments.
>>
>> Just and FYI - I'll be gone until late in this coming week on a
>> backpacking trip in remote Utah. I'll work your suggestions when I
>> get back.
>>
>> With respect to this patch and testing. Yes, I did test and it worked
>> for my situation. I would like to hear from Daniel as I'm not exactly
>> sure what his last change was specific to.
>>
>> Daniel: with the previous patches in this series applied do you still
>> need this quirk for your situation?
>
> It's fine enough for us, since we'll always emit a PXM method for any
> PCI host bridges (but roots), which gets correctly preserved with your
> patches [1]. I don't have reason to suspect northbridges at bus 0,
> devices 0x19 and later in each PCI domain having the wrong NUMA node is
> going to cause any issues.
>
> We could improve things by changing probe_pci_hostbridge to iterate all
> PCI bus domains, but it's probably not a big deal to bail with one
> warning, as we currently see multiple (which would become a lot on most
> of our systems):
>
> [ 3.155974] AMD-Bus: TOM: 00000000d8000000 aka 3456M
> [ 3.160003]
> [ 3.161496] AMD-Bus: WARNING: Failed to find root info for node 0x4,
> link 0x0
> [ 3.168625]
> [ 3.170002]
> [ 3.171492] AMD-Bus: WARNING: Failed to find root info for node 0x4,
> link 0x0
> [ 3.178614] AMD-Bus: WARNING: Failed to find root info for node 0x4,
> link 0x0
> [ 3.180003] AMD-Bus: TOM2: 0000001900000000 aka 102400M
>
> Thanks,
> Daniel
>
> -- [1] (two server NumaConnect system)
>
> # lspci
> 0000:00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890
> Northbridge only dual slot (2x16) PCI-e GFX Hydra part (rev 02)
> 0000:00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890
> PCI to PCI bridge (external gfx1 port B)
> 0000:00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode]
> 0000:00:12.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 0000:00:12.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0 USB OHCI1 Controller
> 0000:00:12.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 0000:00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI0 Controller
> 0000:00:13.1 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0 USB OHCI1 Controller
> 0000:00:13.2 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB EHCI Controller
> 0000:00:14.0 SMBus: Advanced Micro Devices, Inc. [AMD/ATI] SBx00 SMBus
> Controller (rev 3d)
> 0000:00:14.1 IDE interface: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 IDE Controller
> 0000:00:14.3 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 LPC host controller
> 0000:00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00
> PCI to PCI Bridge
> 0000:00:14.5 USB controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 USB OHCI2 Controller
> 0000:00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0000:00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0000:00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0000:00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0000:00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0000:00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0000:00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0000:00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0000:00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0000:00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0000:00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0000:00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0000:00:1a.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0000:00:1a.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0000:00:1a.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0000:00:1a.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0000:00:1a.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0000:00:1a.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0000:00:1b.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0000:00:1b.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0000:00:1b.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0000:00:1b.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0000:00:1b.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0000:00:1b.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0000:00:1c.0 Host bridge: Numascale AS NumaChip N601 (rev 02)
> 0000:00:1c.1 Host bridge: Numascale AS NumaChip N602 (rev 02)
> 0000:01:04.0 VGA compatible controller: Matrox Electronics Systems Ltd.
> MGA G200eW WPCM450 (rev 0a)
> 0000:02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> Network Connection (rev 01)
> 0000:02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
> Network Connection (rev 01)
> 0001:00:00.0 Host bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890
> Northbridge only dual slot (2x16) PCI-e GFX Hydra part (rev 02)
> 0001:00:0d.0 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] RD890
> PCI to PCI bridge (external gfx1 port B)
> 0001:00:11.0 SATA controller: Advanced Micro Devices, Inc. [AMD/ATI]
> SB7x0/SB8x0/SB9x0 SATA Controller [IDE mode]
> 0001:00:14.0 ISA bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00
> SMBus Controller (rev 3d)
> 0001:00:14.4 PCI bridge: Advanced Micro Devices, Inc. [AMD/ATI] SBx00
> PCI to PCI Bridge
> 0001:00:18.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0001:00:18.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0001:00:18.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0001:00:18.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0001:00:18.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0001:00:18.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0001:00:19.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0001:00:19.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0001:00:19.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0001:00:19.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0001:00:19.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0001:00:19.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0001:00:1a.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0001:00:1a.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0001:00:1a.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0001:00:1a.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0001:00:1a.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0001:00:1a.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0001:00:1b.0 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 0
> 0001:00:1b.1 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 1
> 0001:00:1b.2 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 2
> 0001:00:1b.3 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 3
> 0001:00:1b.4 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 4
> 0001:00:1b.5 Host bridge: Advanced Micro Devices, Inc. [AMD] Family 15h
> Processor Function 5
> 0001:00:1c.0 Host bridge: Numascale AS NumaChip N601 (rev 02)
> 0001:00:1c.1 Host bridge: Numascale AS NumaChip N602 (rev 02)
> 0001:02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> Network Connection (rev 01)
> 0001:02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit
> Network Connection (rev 01)
>
> # find /sys/devices/pci* -name numa_node -print -exec cat {} \;
> /sys/devices/pci0000:00/0000:00:00.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:11.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:12.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:12.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:12.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:13.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:13.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:13.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.3/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.4/0000:01:04.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.4/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:14.5/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.3/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.4/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:18.5/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.3/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.4/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:19.5/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:0d.0/0000:02:00.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:0d.0/0000:02:00.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:0d.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.3/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.4/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1a.5/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.1/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.2/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.3/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.4/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1b.5/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1c.0/numa_node
> 0
> /sys/devices/pci0000:00/0000:00:1c.1/numa_node
> 0
> /sys/devices/pci0001:00/0001:00:00.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:11.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:14.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:14.4/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.1/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.2/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.3/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.4/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:18.5/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.1/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.2/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.3/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.4/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:19.5/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:0d.0/0001:02:00.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:0d.0/0001:02:00.1/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:0d.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.1/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.2/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.3/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.4/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1a.5/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.1/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.2/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.3/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.4/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1b.5/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1c.0/numa_node
> 4
> /sys/devices/pci0001:00/0001:00:1c.1/numa_node
> 4
Sorry for late reply. My concern is that removing the
"quirk_amd_nb_node()" will affect the value of "numa_node" of the host
bridge devices (i.e. X:00.[18|19|1a|1b|1c|1d|1e|1f].X). I am not sure
if any code is using this information. But in theory, these host-bridge
devices are not on the same node as where the PCI root complex lives
(e.g. 0 and 4 from the example above).
If we want the "numa_node" to really representing the actual node, then
the quirk has to stay for now. We might need to come up with a
different logic to replace the quirks here, which would automatically
determine the actual node value for these host-bridge devices.
Suravee
Suravee
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists