| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Mon, 26 Jan 2009 15:15:43 -0800
From: "Brandeburg, Jesse" <jesse.brandeburg@...el.com>
To: Paweł Staszewski <pstaszewski@...are.pl>,
Linux Network Development list <netdev@...r.kernel.org>,
"Sosnowski, Maciej" <maciej.sosnowski@...el.com>
CC: "e1000-devel@...ts.sourceforge.net"
<e1000-devel@...ts.sourceforge.net>
Subject: RE: DCA/IOAT problem
Forwarding entire message to include e1000-devel
Pawel Staszewski wrote:
> Hello
>
> Some lspci:
> lspci
> 00:00.0 Host bridge: Intel Corporation Memory Controller Hub (rev 90)
> 00:02.0 PCI bridge: Intel Corporation PCI Express x8 Port 2-3 (rev 90)
> 00:04.0 PCI bridge: Intel Corporation PCI Express x16 Port 4-7 (rev
> 90) 00:08.0 System peripheral: Intel Corporation DMA Engine (rev 90)
> 00:10.0 Host bridge: Intel Corporation FSB Registers (rev 90)
> 00:10.1 Host bridge: Intel Corporation FSB Registers (rev 90)
> 00:10.2 Host bridge: Intel Corporation FSB Registers (rev 90)
> 00:11.0 Host bridge: Intel Corporation Reserved Registers (rev 90)
> 00:13.0 Host bridge: Intel Corporation Reserved Registers (rev 90)
> 00:15.0 Host bridge: Intel Corporation DDR Channel 0 Registers (rev
> 90) 00:16.0 Host bridge: Intel Corporation DDR Channel 1 Registers
> (rev 90) 00:1a.0 USB Controller: Intel Corporation USB UHCI
> Controller #4 (rev 02) 00:1a.7 USB Controller: Intel Corporation USB2
> EHCI Controller #2 (rev 02) 00:1c.0 PCI bridge: Intel Corporation PCI
> Express Port 1 (rev 02) 00:1c.4 PCI bridge: Intel Corporation PCI
> Express Port 5 (rev 02) 00:1c.5 PCI bridge: Intel Corporation PCI
> Express Port 6 (rev 02) 00:1d.0 USB Controller: Intel Corporation USB
> UHCI Controller #1 (rev 02) 00:1d.1 USB Controller: Intel Corporation
> USB UHCI Controller #2 (rev 02) 00:1d.2 USB Controller: Intel
> Corporation USB UHCI Controller #3 (rev 02) 00:1d.7 USB Controller:
> Intel Corporation USB2 EHCI Controller #1 (rev 02) 00:1e.0 PCI
> bridge: Intel Corporation 82801 PCI Bridge (rev 92) 00:1f.0 ISA
> bridge: Intel Corporation LPC Interface Controller (rev 02) 00:1f.2
> IDE interface: Intel Corporation 4 port SATA IDE Controller (rev 02)
> 00:1f.3 SMBus: Intel Corporation SMBus Controller (rev 02) 00:1f.5
> IDE interface: Intel Corporation 2 port SATA IDE Controller (rev 02)
> 01:00.0 Ethernet controller: Intel Corporation Device 10dd (rev 01)
> 04:00.0 Ethernet controller: Intel Corporation 82573E Gigabit
> Ethernet Controller (Copper) (rev 03) 05:00.0 Ethernet controller:
> Intel Corporation 82573L Gigabit Ethernet Controller
> 06:01.0 VGA compatible controller: ATI Technologies Inc ES1000 (rev
> 02)
> and then:
> modprobe ioatdma
>
> dmesg:
> ...
> ...
> IPv4 FIB: Using LC-trie version 0.408
> e1000e 0000:04:00.0: irq 1274 for MSI/MSI-X
> e1000e 0000:04:00.0: irq 1274 for MSI/MSI-X
> e1000e 0000:05:00.0: irq 1273 for MSI/MSI-X
> e1000e 0000:05:00.0: irq 1273 for MSI/MSI-X
> 0000:05:00.0: eth1: Link is Up 1000 Mbps Full Duplex, Flow Control:
> RX/TX dca service started, version 1.4
> ioatdma 0000:00:08.0: can't find IRQ for PCI INT A; probably buggy MP
> table ioatdma 0000:00:08.0: setting latency timer to 64
> ioatdma 0000:00:08.0: Intel(R) I/OAT DMA Engine found, 4 channels,
> device version 0x12, driver version 3.30
> ioatdma 0000:00:08.0: irq 1255 for MSI/MSI-X
> ioatdma 0000:00:08.0: DCA is disabled in BIOS
> ixgbe: eth2: ixgbe_watchdog_task: NIC Link is Up 10 Gbps, Flow
> Control: None e1000e 0000:04:00.0: irq 1274 for MSI/MSI-X
> e1000e 0000:04:00.0: irq 1274 for MSI/MSI-X
>
> ...
> ...
>
>
> lspvi -vvv (for dma engine)
> 00:08.0 System peripheral: Intel Corporation DMA Engine (rev 90)
> Subsystem: Super Micro Computer Inc Device de80
> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx+
> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
> <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 0
> Interrupt: pin A routed to IRQ 1255
> Region 0: Memory at fe700000 (64-bit, non-prefetchable)
> [size=1K] Capabilities: [50] Power Management version 2
> Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
> PME(D0+,D1-,D2-,D3hot+,D3cold+)
> Status: D0 PME-Enable- DSel=0 DScale=0 PME-
> Capabilities: [58] Message Signalled Interrupts: Mask- 64bit-
> Queue=0/0 Enable+
> Address: feeff00c Data: 41b2
> Capabilities: [6c] Express (v1) Root Complex Integrated
> Endpoint, MSI 00
> DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s
> <64ns, L1 <1us
> ExtTag- RBE- FLReset-
> DevCtl: Report errors: Correctable- Non-Fatal- Fatal+
> Unsupported-
> RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop+
> MaxPayload 128 bytes, MaxReadReq 128 bytes
> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
> AuxPwr- TransPend-
> LnkCap: Port #0, Speed unknown, Width x0, ASPM
> unknown, Latency L0 <64ns, L1 <1us
> ClockPM- Suprise- LLActRep- BwNot-
> LnkCtl: ASPM Disabled; Disabled- Retrain- CommClk-
> ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
> LnkSta: Speed unknown, Width x0, TrErr- Train-
> SlotClk- DLActive- BWMgmt- ABWMgmt-
> Kernel driver in use: ioatdma
>
>
> cat /proc/interrupts
> CPU0 CPU1 CPU2 CPU3 CPU4
> CPU5 CPU6 CPU7
> 0: 81 6 9 7 5
> 3 9 5 IO-APIC-edge timer
> 1: 1 0 0 0 1
> 0 0 0 IO-APIC-edge i8042
> 2: 0 0 0 0 0
> 0 0 0 XT-PIC-XT cascade
> 14: 901 857 867 864 708
> 674 693 692 IO-APIC-edge ata_piix
> 15: 686 728 691 703 826
> 897 876 872 IO-APIC-edge ata_piix
> 18: 0 0 0 0 0
> 0 0 0 IO-APIC-fasteoi ata_piix
> 1255: 1 0 1 0 1
> 0 1 1 PCI-MSI-edge ioat-msi
> 1256: 0 0 0 0 0
> 0 0 0 PCI-MSI-edge eth2:lsc
> 1257: 857 720 1176 802 960
> 751 706 841 PCI-MSI-edge eth2:v15-Tx
> 1258: 958 751 706 841 1176
> 802 720 859 PCI-MSI-edge eth2:v14-Tx
> 1259: 1176 802 720 859 706
> 841 751 958 PCI-MSI-edge eth2:v13-Tx
> 1260: 706 842 751 959 720
> 858 802 1175 PCI-MSI-edge eth2:v12-Tx
> 1261: 720 858 802 1174 751
> 960 842 706 PCI-MSI-edge eth2:v11-Tx
> 1262: 751 960 842 706 802
> 1174 858 720 PCI-MSI-edge eth2:v10-Tx
> 1263: 802 1174 858 720 842
> 706 960 751 PCI-MSI-edge eth2:v9-Tx
> 1264: 842 707 960 751 859
> 719 1173 802 PCI-MSI-edge eth2:v8-Tx
> 1265: 814186 813591 813713 814872 814537
> 816002 812892 814292 PCI-MSI-edge eth2:v7-Rx
> 1266: 626856 626688 627886 627507 628067
> 625730 626561 626521 PCI-MSI-edge eth2:v6-Rx
> 1267: 592474 591312 591741 592107 592214
> 593218 591271 591386 PCI-MSI-edge eth2:v5-Rx
> 1268: 558076 559008 557856 558677 557894
> 556703 557997 558879 PCI-MSI-edge eth2:v4-Rx
> 1269: 723457 725019 723746 724688 723165
> 724716 724037 725136 PCI-MSI-edge eth2:v3-Rx
> 1270: 553387 554096 553204 552603 553078
> 553587 553838 553769 PCI-MSI-edge eth2:v2-Rx
> 1271: 603820 605323 605368 602989 604948
> 604887 606911 604745 PCI-MSI-edge eth2:v1-Rx
> 1272: 800828 798049 799471 799703 799183
> 798322 799220 798254 PCI-MSI-edge eth2:v0-Rx
> 1273: 2868 2894 2970 2808 2863
> 2803 3167 2860 PCI-MSI-edge eth1
> 1274: 873 856 882 924 933
> 904 864 943 PCI-MSI-edge eth0
> NMI: 0 0 0 0 0
> 0 0 0 Non-maskable interrupts
> LOC: 40527 41783 40049 40903 42180
> 39477 49021 47628 Local timer interrupts
> RES: 858 58 232 283 89
> 97 284 435 Rescheduling interrupts
> CAL: 52 101 99 88 96
> 93 92 58 Function call interrupts
> TLB: 143 133 486 452 141
> 159 396 429 TLB shootdowns
> SPU: 0 0 0 0 0
> 0 0 0 Spurious interrupts
> ERR: 0
> MIS: 0
>
>
> cat /proc/cpuinfo
> cat /proc/cpuinfo
> processor : 0
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 0
> cpu cores : 4
> apicid : 0
> initial apicid : 0
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 5999.79
> clflush size : 64
> power management:
>
> processor : 1
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 1
> siblings : 4
> core id : 0
> cpu cores : 4
> apicid : 4
> initial apicid : 4
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6065.40
> clflush size : 64
> power management:
>
> processor : 2
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 1
> cpu cores : 4
> apicid : 1
> initial apicid : 1
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.46
> clflush size : 64
> power management:
>
> processor : 3
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 1
> siblings : 4
> core id : 1
> cpu cores : 4
> apicid : 5
> initial apicid : 5
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.47
> clflush size : 64
> power management:
>
> processor : 4
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 2
> cpu cores : 4
> apicid : 2
> initial apicid : 2
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.43
> clflush size : 64
> power management:
>
> processor : 5
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 1
> siblings : 4
> core id : 2
> cpu cores : 4
> apicid : 6
> initial apicid : 6
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.50
> clflush size : 64
> power management:
>
> processor : 6
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 0
> siblings : 4
> core id : 3
> cpu cores : 4
> apicid : 3
> initial apicid : 3
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.43
> clflush size : 64
> power management:
>
> processor : 7
> vendor_id : GenuineIntel
> cpu family : 6
> model : 23
> model name : Intel(R) Xeon(R) CPU X5450 @ 3.00GHz
> stepping : 6
> cpu MHz : 2999.898
> cache size : 6144 KB
> physical id : 1
> siblings : 4
> core id : 3
> cpu cores : 4
> apicid : 7
> initial apicid : 7
> fdiv_bug : no
> hlt_bug : no
> f00f_bug : no
> coma_bug : no
> fpu : yes
> fpu_exception : yes
> cpuid level : 10
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx
> lm constant_tsc arch_perfmon pebs bts pni dtes64 monitor ds_cpl vmx
> est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 lahf_lm tpr_shadow vnmi
> flexpriority bogomips : 6000.47
> clflush size : 64
> power management:
>
>
>
>
> I have enabled DCA in my Bios.
> So why i have info:
> ioatdma 0000:00:08.0: DCA is disabled in BIOS
Probably because the BIOS is not working correctly or there is a bug in
dca. Is that a supermicro machine? Can you include the output of
dmidecode? Are you running the latest BIOS? What are you trying to
achieve? DCA or IOAT DMA copy offload (two separate features)
>
> and:
> ls -la /sys/class/dca/
> total 0
> drwxr-xr-x 2 root root 0 Jan 26 16:21 .
> drwxr-xr-x 33 root root 0 Jan 26 16:14 ..
>
> cat /sys/class/dma/dma0chan0/in_use
> 0
> cat /sys/class/dma/dma0chan1/in_use
> 1
> cat /sys/class/dma/dma0chan2/in_use
> 0
> cat /sys/class/dma/dma0chan3/in_use
> 0
>
>
> Why DCA is not working (or working but no info about in ioat)
The DMA channels (copy offload) are different than the DCA feature. DCA
involves front side bus hints sent from the adapter to the CPU handling
the interrupt, where DMA channels are for copy offload from the processor
to the independent data movers (DMA channels)
Download attachment "smime.p7s" of type "application/x-pkcs7-signature" (6703 bytes)
Powered by blists - more mailing lists