| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <201005151026.20356.carsten.aulbert@aei.mpg.de>
Date: Sat, 15 May 2010 10:26:17 +0200
From: Carsten Aulbert <carsten.aulbert@....mpg.de>
To: "\"Brandeburg, Jesse\"" <jesse.brandeburg@...el.com>
Cc: Tejun Heo <tj@...nel.org>,
"Ronciak, John" <john.ronciak@...el.com>,
Henning Fehrmann <henning.fehrmann@....mpg.de>,
"Kirsher, Jeffrey T" <jeffrey.t.kirsher@...el.com>,
"Allan, Bruce W" <bruce.w.allan@...el.com>,
"Waskiewicz Jr, Peter P" <peter.p.waskiewicz.jr@...el.com>,
"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
Matt Mackall <mpm@...enic.com>,
e1000-devel@...ts.sourceforge.net
Subject: Re: loosing IPMI-card by loading netconsole
Hi all,
I'll try to gather some information:
The system under question is Supermicro PSDML-LN2+ based, eth0 is a 82573E
while eth1 is a 82573L. IPMI is piggy-bagged onto eth0, eth1 is our "data
network" running with large jumbo frames:
# ifconfig
eth0 Link encap:Ethernet HWaddr 00:30:48:96:e1:e2
inet addr:172.26.1.26 Bcast:172.31.255.255 Mask:255.240.0.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:673128 errors:0 dropped:0 overruns:0 frame:0
TX packets:104281 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:100
RX bytes:46271457 (44.1 MiB) TX bytes:8238586 (7.8 MiB)
Memory:ee100000-ee120000
eth1 Link encap:Ethernet HWaddr 00:30:48:96:e1:e3
inet addr:10.10.1.26 Bcast:10.255.255.255 Mask:255.0.0.0
UP BROADCAST RUNNING MULTICAST MTU:9000 Metric:1
RX packets:17660268 errors:0 dropped:72239 overruns:0 frame:0
TX packets:10941117 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:86507558735 (80.5 GiB) TX bytes:7011613344 (6.5 GiB)
Memory:ee200000-ee220000
# ethtool -i eth0
driver: e1000e
version: 1.0.2-k2
firmware-version: 0.15-4
bus-info: 0000:0d:00.0
ethtool -i eth1
driver: e1000e
version: 1.0.2-k2
firmware-version: 0.5-7
bus-info: 0000:0e:00.0
lscpi:
0d:00.0 0200: 8086:108c (rev 03)
Subsystem: 15d9:108c
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 29
Region 0: Memory at ee100000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at 4000 [size=32]
Capabilities: [c8] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable+
Address: 00000000fee0f00c Data: 41c1
Capabilities: [e0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns,
L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr+
TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM unknown,
Latency L0 <128ns, L1 <64us
ClockPM- Suprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [140] Device Serial Number e2-e1-96-ff-ff-48-30-00
Kernel driver in use: e1000e
Kernel modules: e1000e
0e:00.0 0200: 8086:109a
Subsystem: 15d9:109a
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
Stepping- SERR+ FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 30
Region 0: Memory at ee200000 (32-bit, non-prefetchable) [size=128K]
Region 2: I/O ports at 5000 [size=32]
Capabilities: [c8] Power Management version 2
Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D0 PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [d0] Message Signalled Interrupts: Mask- 64bit+
Queue=0/0 Enable+
Address: 00000000fee0f00c Data: 41b1
Capabilities: [e0] Express (v1) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s <512ns,
L1 <64us
ExtTag- AttnBtn- AttnInd- PwrInd- RBE- FLReset-
DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+
Unsupported+
RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
MaxPayload 128 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- UncorrErr+ FatalErr- UnsuppReq+ AuxPwr+
TransPend-
LnkCap: Port #0, Speed 2.5GT/s, Width x1, ASPM unknown,
Latency L0 <128ns, L1 <64us
ClockPM+ Suprise- LLActRep- BwNot-
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain-
CommClk-
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s, Width x1, TrErr- Train- SlotClk+
DLActive- BWMgmt- ABWMgmt-
Capabilities: [100] Advanced Error Reporting <?>
Capabilities: [140] Device Serial Number e3-e1-96-ff-ff-48-30-00
Kernel driver in use: e1000e
Kernel modules: e1000e
***************************************************************************
OK, now the detailed parts:
IPMI is currently blocked, i.e. I can query the card locally via the kernel
module, but not remotely. netconsole is loaded and working via eth1.
# ethtool -S eth0
NIC statistics:
rx_packets: 673400
tx_packets: 104323
rx_bytes: 48982938
tx_bytes: 8659194
rx_broadcast: 619125
tx_broadcast: 3
rx_multicast: 2088
tx_multicast: 0
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 2088
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 48982938
rx_csum_offload_good: 76752
rx_csum_offload_errors: 0
rx_header_split: 0
alloc_rx_buff_failed: 0
tx_smbus: 2
rx_smbus: 594496
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
n0126:/build/ethregs-1.7.2# ethtool -S eth1
NIC statistics:
rx_packets: 17664653
tx_packets: 10941353
rx_bytes: 86578358008
tx_bytes: 7055320722
rx_broadcast: 4761580
tx_broadcast: 1252
rx_multicast: 0
tx_multicast: 0
rx_errors: 0
tx_errors: 0
tx_dropped: 0
multicast: 0
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 0
rx_frame_errors: 0
rx_no_buffer_count: 1985
rx_missed_errors: 72239
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 1920
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 127295
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 86578358008
rx_csum_offload_good: 12907089
rx_csum_offload_errors: 0
rx_header_split: 750356
alloc_rx_buff_failed: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0
rx_dma_failed: 0
tx_dma_failed: 0
ethreg result attached
******************************************************************************
After rebooting without netconsole the IPMI card was still not working, using
the echo 1 > ... /remove command from Henning's original email suddenly made
IPMI working again.
After that I rebooted the machine while watching the progress via ipmitool's
sol activate, however during the boot sequence suddenly the kernel output
slowed to a crawl. About 1-2 minutes later remote IPMI broke down again.
netconsole was NOT loaded, thus I don't think that this is really related.
OK, how to proceed?
e.g. how can I re-enable the interface once I issued
echo 1 > /sys/devices/pci0000:00/0000:00:1c.4/0000:0d:00.0/remove
Cheers
Carsten
View attachment "ethregs.IPMInotworkingNOnetconsole" of type "text/plain" (41597 bytes)
View attachment "ethregs.IPMInotworking" of type "text/plain" (41598 bytes)
Powered by blists - more mailing lists