netdev - Re: 3.19+: (and quite probably earlier) VIA Rhine hanging under high network load, yet again: redux

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <87mw2mw3w7.fsf@spindle.srvr.nix>
Date:	Mon, 06 Apr 2015 00:29:12 +0100
From:	Nix <nix@...eri.org.uk>
To:	Francois Romieu <romieu@...zoreil.com>
Cc:	rl@...lgate.ch, Bjarke Istrup Pedersen <gurligebis@...too.org>,
	"David S. Miller" <davem@...emloft.net>,
	Linux-Netdev <netdev@...r.kernel.org>
Subject: Re: 3.19+: (and quite probably earlier) VIA Rhine hanging under high network load, yet again: redux

On 6 Apr 2015, Francois Romieu outgrape:

> Nix <nix@...eri.org.uk> :
> [...]
>> Gross or not, it seems to work: I've loaded it enough to crash it half a
>> dozen times, and not a crash. However, the rx_dropped stats on the link
>> aren't going up, so maybe I've just been lucky.
>
> Rx descriptors are now recycled as soon as they are processed whereas the
> driver used to perform a complete processing batch before recycling any
> descriptor. It could make a huge difference.

Ah, of course, nothing bounds rx rates :( I was stupidly thinking the
TX_RING_SIZE / TX_QUEUE_LEN gap would help us, but of course that's on
the other side. I just shouldn't read code when thick with cold, I make
really stupid thinkos... tx != rx dammit, it's not like they even share
much code in this driver, with rx being run out of napipoll and tx still
being direct...

(I'm still surprised a 64-entry RX ring can run us out of memory,
though: 64 * 1500 isn't that big, even for atomic allocations...)

> The pre-patch rx batch recycling did not include any barrier between
> rp->rx_ring[entry].addr and rp->rx_ring[entry].rx_status updates to
> enforce the ordering.

I bet that's the crucial part. At high rx rates in the pre-patch driver,
you fill up the ring and then lose that race, and disaster ensues.

> Whatever the outcome I'll have to clean my mess though.

Your mess has a) fixed the problem and b) fixed the problem *during the
easter break*. Major kudos.
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html