lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Wed, 13 Jan 2010 12:55:31 -0800
From:	"Matt Carlson" <mcarlson@...adcom.com>
To:	"Peter Rabbitson" <rabbit+list@...bit.us>
cc:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"Matthew Carlson" <mcarlson@...adcom.com>,
	"Michael Chan" <mchan@...adcom.com>
Subject: Re: Reproducible corruption with SG offloading

While I think about this a little more can you try two things?

1) If you can, try moving the NIC to a different slot.

2) Try the following patch:

diff --git a/drivers/net/tg3.c b/drivers/net/tg3.c
index 7f82b02..fa4c52e 100644
--- a/drivers/net/tg3.c
+++ b/drivers/net/tg3.c
@@ -7645,6 +7645,9 @@ static int tg3_reset_hw(struct tg3 *tp, int reset_phy)
 	 */
 	tp->grc_mode |= GRC_MODE_NO_TX_PHDR_CSUM;
 
+	if (GET_ASIC_REV(tp->pci_chip_rev_id) == ASIC_REV_5701)
+		tp->grc_mode |= GRC_MODE_FORCE_PCI32BIT;
+
 	tw32(GRC_MODE,
 	     tp->grc_mode |
 	     (GRC_MODE_IRQ_ON_MAC_ATTN | GRC_MODE_HOST_STACKUP));



On Mon, Jan 11, 2010 at 04:22:51AM -0800, Peter Rabbitson wrote:
> I have the following card:
> 
> 02:02.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit Ethernet (rev 15)
> 	Subsystem: Compaq Computer Corporation NC7770 Gigabit Server Adapter (PCI-X, 10/100/1000-T)
> 	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr+ Stepping- SERR+ FastB2B- DisINTx-
> 	Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> 	Latency: 64 (16000ns min), Cache Line Size: 64 bytes
> 	Interrupt: pin A routed to IRQ 24
> 	Region 0: Memory at fc5e0000 (64-bit, non-prefetchable) [size=64K]
> 	[virtual] Expansion ROM at fc5d0000 [disabled] [size=64K]
> 	Capabilities: [40] PCI-X non-bridge device
> 		Command: DPERE- ERO- RBC=512 OST=1
> 		Status: Dev=02:02.1 64bit+ 133MHz+ SCD- USC- DC=simple DMMRBC=512 DMOST=1 DMCRS=8 RSCEM- 266MHz- 533MHz-
> 	Capabilities: [48] Power Management version 2
> 		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot+,D3cold+)
> 		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=1 PME-
> 	Capabilities: [50] Vital Product Data
> 		Product Name: HP NC7770 Gigabit Server Adapter
> 		Read-only fields:
> 			[PN] Part number: 284685-003
> 			[EC] Engineering changes: 0G
> 			[SN] Serial number: WEURE9FB68
> 			[MN] Manufacture ID: 30 45 31 31
> 			[RV] Reserved: checksum bad, 47 byte(s) reserved
> 		Read/write fields:
> 			[YA] Asset tag: Hewlett Packard
> 			[RW] Read-write area: 103 byte(s) free
> 		End
> 	Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
> 		Address: 00312230088fb704  Data: 0000
> 	Kernel driver in use: tg3
> 
> When confronted with very specific repeatable data streams the card eventually
> starts setting garbage down the wire. If I do ethtool -k <ifname> sg off the
> problem disappears. Let me know if ou are interested in tracking this down,
> (whether software or hardware) otherwise I'll throw the card away and go with
> something else.
> 
> Please CC me as I am not subscribed to netdev@
> 
> This is my minimal test case with relevant data:
> 
> Program
> ============
> 
> #!/usr/bin/perl
> use warnings;
> use strict;
> 
> use DBI;
> my $dbh = DBI->connect ('dbi:mysql:database=tst;host=192.168.5.2', 'stress', 123, { RaiseError => 1});
> 
> for (1 .. 92) {
>   $dbh->selectall_arrayref(q|SELECT `alias` FROM dbmail_aliases WHERE `alias` like "spam@...mple.com"|);
> }
> 
> tcpdump at the machine with the card in question
> ======================================
> 
> 00000000  d4 c3 b2 a1 02 00 04 00  00 00 00 00 00 00 00 00  |................|
> 00000010  ff ff 00 00 01 00 00 00  7d 16 4b 4b b2 1a 06 00  |........}.KK....|
> 00000020  4a 00 00 00 4a 00 00 00  00 30 48 67 c2 1c 00 11  |J...J....0Hg....|
> 00000030  0a e9 fb 68 08 00 45 00  00 3c e9 c3 40 00 40 06  |...h..E..<..@.@.|
> 00000040  c5 a4 c0 a8 05 01 c0 a8  05 02 95 30 0c ea 31 99  |...........0..1.|
> 00000050  f8 84 00 00 00 00 a0 02  16 d0 c5 ed 00 00 02 04  |................|
> 00000060  05 b4 04 02 08 0a 02 70  11 47 00 00 00 00 01 03  |.......p.G......|
> 00000070  03 06 7d 16 4b 4b 5c 1b  06 00 4a 00 00 00 4a 00  |..}.KK\...J...J.|
> 00000080  00 00 00 11 0a e9 fb 68  00 30 48 67 c2 1c 08 00  |.......h.0Hg....|
> 00000090  45 00 00 3c 00 00 40 00  40 06 af 68 c0 a8 05 02  |E..<..@.@.......|
> 000000a0  c0 a8 05 01 0c ea 95 30  c8 89 90 8c 31 99 f8 85  |.......0....1...|
> 000000b0  a0 12 16 a0 f4 28 00 00  02 04 05 b4 04 02 08 0a  |.....(..........|
> 000000c0  06 8b 72 41 02 70 11 47  01 03 03 07 7d 16 4b 4b  |..rA.p.G....}.KK|
> 000000d0  80 1b 06 00 42 00 00 00  42 00 00 00 00 30 48 67  |....B...B....0Hg|
> 000000e0  c2 1c 00 11 0a e9 fb 68  08 00 45 00 00 34 e9 c4  |.......h..E..4..|
> 000000f0  40 00 40 06 c5 ab c0 a8  05 01 c0 a8 05 02 95 30  |@.@............0|
> 00000100  0c ea 31 99 f8 85 c8 89  90 8d 80 10 00 5c 39 39  |..1..........\99|
> 00000110  00 00 01 01 08 0a 02 70  11 47 06 8b 72 41 7d 16  |.......p.G..rA}.|
> 00000120  4b 4b ad 1c 06 00 80 00  00 00 80 00 00 00 00 11  |KK..............|
> 00000130  0a e9 fb 68 00 30 48 67  c2 1c 08 00 45 08 00 72  |...h.0Hg....E..r|
> 00000140  e2 cc 40 00 40 06 cc 5d  c0 a8 05 02 c0 a8 05 01  |..@.@..]........|
> 00000150  0c ea 95 30 c8 89 90 8d  31 99 f8 85 80 18 00 2e  |...0....1.......|
> 00000160  79 db 00 00 01 01 08 0a  06 8b 72 41 02 70 11 47  |y.........rA.p.G|
> 00000170  3a 00 00 00 0a 35 2e 31  2e 34 31 2d 33 2d 6c 6f  |:....5.1.41-3-lo|
> 00000180  67 00 80 00 00 00 5d 2d  66 3a 2e 75 29 66 00 ff  |g.....]-f:.u)f..|
> 00000190  f7 21 02 00 00 00 00 00  00 00 00 00 00 00 00 00  |.!..............|
> 000001a0  00 48 72 2a 74 2e 24 69  67 48 72 29 6d 00 7d 16  |.Hr*t.$igHr)m.}.|
> 000001b0  4b 4b db 1c 06 00 42 00  00 00 42 00 00 00 00 30  |KK....B...B....0|
> 000001c0  48 67 c2 1c 00 11 0a e9  fb 68 08 00 45 08 00 34  |Hg.......h..E..4|
> 000001d0  e9 c5 40 00 40 06 c5 a2  c0 a8 05 01 c0 a8 05 02  |..@.@...........|
> 000001e0  95 30 0c ea 31 99 f8 85  c8 89 90 cb 80 10 00 5c  |.0..1..........\|
> 000001f0  38 fb 00 00 01 01 08 0a  02 70 11 47 06 8b 72 41  |8........p.G..rA|
> 00000200  7d 16 4b 4b 3d 30 06 00  86 00 00 00 86 00 00 00  |}.KK=0..........|
> 00000210  00 30 48 67 c2 1c 00 11  0a e9 fb 68 08 00 45 08  |.0Hg.......h..E.|
> 00000220  00 78 e9 c6 40 00 40 06  c5 5d c0 a8 05 01 c0 a8  |.x..@.@..]......|
> 00000230  05 02 95 30 0c ea 31 99  f8 85 c8 89 90 cb 80 18  |...0..1.........|
> 00000240  00 5c 8b be 00 00 01 01  08 0a 02 70 11 48 06 8b  |.\.........p.H..|
> 00000250  72 41 40 00 00 01 8f a2  02 00 00 00 00 40 08 00  |rA@..........@..|
> 00000260  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
> 00000270  00 00 00 00 00 00 73 74  72 65 73 73 00 14 5d 16  |......stress..].|
> 00000280  8c 5d 41 ff 26 31 fc 5a  db fa 42 27 4a a1 6b 16  |.]A.&1.Z..B'J.k.|
> 00000290  1a 28 74 73 74 00 7d 16  4b 4b b8 30 06 00 42 00  |.(tst.}.KK.0..B.|
> 000002a0  00 00 42 00 00 00 00 11  0a e9 fb 68 00 30 48 67  |..B........h.0Hg|
> 000002b0  c2 1c 08 00 45 08 00 34  e2 cd 40 00 40 06 cc 9a  |....E..4..@.@...|
> 000002c0  c0 a8 05 02 c0 a8 05 01  0c ea 95 30 c8 89 90 cb  |...........0....|
> 000002d0  31 99 f8 c9 80 10 00 2e  38 e2 00 00 01 01 08 0a  |1......8.......|
> 000002e0  06 8b 72 43 02 70 11 48  7d 16 4b 4b f5 30 06 00  |..rC.p.H}.KK.0..|
> 000002f0  4d 00 00 00 4d 00 00 00  00 11 0a e9 fb 68 00 30  |M...M........h.0|
> 00000300  48 67 c2 1c 08 00 45 08  00 3f e2 ce 40 00 40 06  |Hg....E..?..@.@.|
> 00000310  cc 8e c0 a8 05 02 c0 a8  05 01 0c ea 95 30 c8 89  |.............0..|
> 00000320  90 cb 31 99 f8 c9 80 18  00 2e 31 cb 00 00 01 01  |..1.......1.....|
> 00000330  08 0a 06 8b 72 43 02 70  11 48 07 00 00 02 00 00  |....rC.p.H......|
> 00000340  00 02 00 00 00 7d 16 4b  4b b9 31 06 00 57 00 00  |.....}.KK.1..W..|
> 00000350  00 57 00 00 00 00 30 48  67 c2 1c 00 11 0a e9 fb  |.W....0Hg.......|
> 00000360  68 08 00 45 08 00 49 e9  c7 40 00 40 06 c5 8b c0  |h..E..I..@.@....|
> 00000370  a8 05 01 c0 a8 05 02 95  30 0c ea 31 99 f8 c9 c8  |........0..1....|
> 00000380  89 90 d6 80 18 00 5c 8b  8f 00 00 01 01 08 0a 02  |......\.........|
> 00000390  70 11 49 06 8b 72 43 11  00 00 00 03 73 65 74 20  |p.I..rC.....set |
> 000003a0  61 75 74 6f 63 6f 6d 6d  69 74 3d 31 7d 16 4b 4b  |autocommit=1}.KK|
> 000003b0  a1 32 06 00 4d 00 00 00  4d 00 00 00 00 11 0a e9  |.2..M...M.......|
> 000003c0  fb 68 00 30 48 67 c2 1c  08 00 45 08 00 3f e2 cf  |.h.0Hg....E..?..|
> 000003d0  40 00 40 06 cc 8d c0 a8  05 02 c0 a8 05 01 0c ea  |@.@.............|
> 000003e0  95 30 c8 89 90 d6 31 99  f8 de 80 18 00 2e 31 ab  |.0....1.......1.|
> 000003f0  00 00 01 01 08 0a 06 8b  72 43 02 70 11 49 07 00  |........rC.p.I..|
> 00000400  00 01 00 00 00 02 00 00  00 7d 16 4b 4b 88 33 06  |.........}.KK.3.|
> 00000410  00 8f 00 00 00 8f 00 00  00 00 30 48 67 c2 1c 00  |..........0Hg...|
> 00000420  11 0a e9 fb 68 08 00 45  08 00 81 e9 c8 40 00 40  |....h..E.....@.@|
> 00000430  06 c5 52 c0 a8 05 01 c0  a8 05 02 95 30 0c ea 31  |..R.........0..1|
> 00000440  99 f8 de c8 89 90 e1 80  18 00 5c 8b c7 00 00 01  |..........\.....|
> 00000450  01 08 0a 02 70 11 49 06  8b 72 43 49 00 00 00 03  |....p.I..rCI....|
> 00000460  53 45 4c 45 43 54 20 60  61 6c 69 61 73 60 20 46  |SELECT `alias` F|
> 00000470  52 4f 4d 20 64 62 6d 61  69 6c 5f 61 6c 69 61 73  |ROM dbmail_alias|
> 00000480  65 73 20 57 48 45 52 45  20 60 61 6c 69 61 73 60  |es WHERE `alias`|
> 00000490  20 6c 69 6b 65 20 22 73  70 61 6d 40 65 78 61 6d  | like "spam@...m|
> 000004a0  70 6c 65 2e 63 6f 6d 22  7d 16 4b 4b 3c 34 06 00  |ple.com"}.KK<4..|
> 000004b0  9c 00 00 00 9c 00 00 00  00 11 0a e9 fb 68 00 30  |.............h.0|
> 000004c0  48 67 c2 1c 08 00 45 08  00 8e e2 d0 40 00 40 06  |Hg....E.....@.@.|
> 000004d0  cc 3d c0 a8 05 02 c0 a8  05 01 0c ea 95 30 c8 89  |.=...........0..|
> 000004e0  90 e1 31 99 f9 2b 80 18  00 2e b4 4c 00 00 01 01  |..1..+.....L....|
> 000004f0  08 0a 06 8b 72 43 02 70  11 49 01 00 00 01 01 3f  |....rC.p.I.....?|
> 00000500  00 00 02 03 64 65 66 03  74 73 74 0e 64 62 6d 61  |....def.tst.dbma|
> 00000510  69 6c 5f 61 6c 69 61 73  65 73 0e 64 62 6d 61 69  |il_aliases.dbmai|
> 00000520  6c 5f 61 6c 69 61 73 65  73 05 61 6c 69 61 73 05  |l_aliases.alias.|
> 00000530  61 6c 69 61 73 0c 08 00  64 00 00 00 fd 00 00 00  |alias...d.......|
> 00000540  00 00 05 00 00 03 fe 00  00 22 00 05 00 00 04 fe  |........."......|
> 00000550  00 00 22 00 7d 16 4b 4b  e0 34 06 00 8f 00 00 00  |..".}.KK.4......|
> 00000560  8f 00 00 00 00 30 48 67  c2 1c 00 11 0a e9 fb 68  |.....0Hg.......h|
> 00000570  08 00 45 08 00 81 e9 c9  40 00 40 06 c5 51 c0 a8  |..E.....@.@.....|
> 00000580  05 01 c0 a8 05 02 95 30  0c ea 31 99 f9 2b c8 89  |.......0..1..+..|
> 00000590  91 3b 80 18 00 5c 8b c7  00 00 01 01 08 0a 02 70  |.;...\.........p|
> 000005a0  11 49 06 8b 72 43 49 00  00 00 03 53 45 4c 45 43  |.I..rCI....SELEC|
> 000005b0  54 20 60 61 6c 69 61 73  60 20 46 52 4f 4d 20 64  |T `alias` FROM d|
> 000005c0  62 6d 61 69 6c 5f 61 6c  69 61 73 65 73 20 57 48  |bmail_aliases WH|
> 000005d0  45 52 45 20 60 61 6c 69  61 73 60 20 6c 69 6b 65  |ERE `alias` like|
> 000005e0  20 22 73 70 61 6d 40 65  78 61 6d 70 6c 65 2e 63  | "spam@...mple.c|
> 000005f0  6f 6d 22 7d 16 4b 4b 70  35 06 00 9c 00 00 00 9c  |om"}.KKp5.......|
> ......
> the select statement/response repeated verbatim 90+ times
> ......
> 00007a00  06 8b 72 49 49 00 00 00  03 53 45 4c 45 43 54 20  |..rII....SELECT |
> 00007a10  60 61 6c 69 61 73 60 20  46 52 4f 4d 20 64 62 6d  |`alias` FROM dbm|
> 00007a20  61 69 6c 5f 61 6c 69 61  73 65 73 20 57 48 45 52  |ail_aliases WHER|
> 00007a30  45 20 60 61 6c 69 61 73  60 20 6c 69 6b 65 20 22  |E `alias` like "|
> 00007a40  73 70 61 6d 40 65 78 61  6d 70 6c 65 2e 63 6f 6d  |spam@...mple.com|
> 00007a50  22 7d 16 4b 4b d2 98 06  00 f3 00 00 00 f3 00 00  |"}.KK...........|
> 00007a60  00 00 11 0a e9 fb 68 00  30 48 67 c2 1c 08 00 45  |......h.0Hg....E|
> 00007a70  08 00 e5 e3 2b 40 00 40  06 cb 8b c0 a8 05 02 c0  |....+@.@........|
> 00007a80  a8 05 01 0c ea 95 30 c8  89 b0 df 31 9a 14 8a 80  |......0....1....|
> 00007a90  18 00 2e 43 e0 00 00 01  01 08 0a 06 8b 72 49 02  |...C.........rI.|
> ....
> followed by the server error message
> 
> What the other side actually sees (note byte 7a50)
> ======================
> ...
> 00007a00  06 8b 72 49 49 00 00 00  03 53 45 4c 45 43 54 20  |..rII....SELECT |
> 00007a10  60 61 6c 69 61 73 60 20  46 52 4f 4d 20 64 62 6d  |`alias` FROM dbm|
> 00007a20  61 69 6c 5f 61 6c 69 61  73 65 73 20 57 48 45 52  |ail_aliases WHER|
> 00007a30  45 20 60 61 6c 69 61 73  60 20 6c 69 6b 65 20 22  |E `alias` like "|
> 00007a40  73 70 61 6d 40 65 78 61  6d 70 6c 65 2e 63 6f 6d  |spam@...mple.com|
> 00007a50  6d 7d 16 4b 4b 3d 66 06  00 f3 00 00 00 f3 00 00  |m}.KK=f.........|
> 00007a60  00 00 11 0a e9 fb 68 00  30 48 67 c2 1c 08 00 45  |......h.0Hg....E|
> 00007a70  08 00 e5 e3 2b 40 00 40  06 cb 8b c0 a8 05 02 c0  |....+@.@........|
> 00007a80  a8 05 01 0c ea 95 30 c8  89 b0 df 31 9a 14 8a 80  |......0....1....|
> 00007a90  18 00 2e 8c 2b 00 00 01  01 08 0a 06 8b 72 49 02  |....+........rI.|
> 
> 
> 

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists