netdev - MAX_SKB_FRAGS and GRO

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100208100306.GQ32246@kryten>
Date:	Mon, 8 Feb 2010 21:03:07 +1100
From:	Anton Blanchard <anton@...ba.org>
To:	herbert@...dor.apana.org.au
Cc:	Jeff Kirsher <jeffrey.t.kirsher@...el.com>,
	Jesse Brandeburg <jesse.brandeburg@...el.com>,
	Bruce Allan <bruce.w.allan@...el.com>,
	Alex Duyck <alexander.h.duyck@...el.com>,
	PJ Waskiewicz <peter.p.waskiewicz.jr@...el.com>,
	John Ronciak <john.ronciak@...el.com>, divy@...lsio.com,
	netdev@...r.kernel.org
Subject: MAX_SKB_FRAGS and GRO

Hi Herbert,

I was looking through the hardware GRO support in various drivers and I think
we have a couple of issues with PAGE_SIZE > 4k. For example, if we have a 64kB
page size then MAX_SKB_FRAGS ends up as 3:

#define MAX_SKB_FRAGS (65536/PAGE_SIZE + 2)

This should be fine for hardware and software GSO, but we encounter issues
with hardware GRO (not sure about software GRO).

In the ixgbe case we use MAX_SKB_FRAGS to program the max number of GRO
descriptors, even though we assemble GRO packets using ->frag_list:

#if (MAX_SKB_FRAGS > 16)
                rscctrl |= IXGBE_RSCCTL_MAXDESC_16;
#elif (MAX_SKB_FRAGS > 8)
                rscctrl |= IXGBE_RSCCTL_MAXDESC_8;
#elif (MAX_SKB_FRAGS > 4)
                rscctrl |= IXGBE_RSCCTL_MAXDESC_4;
#else
                rscctrl |= IXGBE_RSCCTL_MAXDESC_1;
#endif

With MAX_SKB_FRAGS = 3 it looks like we only allow 1 GRO descriptor, and since
the card can only do DMAs of 16kB that will be our maximum GRO packet.

I don't have a hardware GRO capable ixgbe to try this out yet, but I think
we want to do something like the attached (completely untested) patch, dividing
GSO_MAX_SIZE by our per packet maximum.

In the cxgb3 case we build GRO packets via ->frags. (FYI I had a look through
the driver but I wasn't able to see where it caps GRO assembly to
MAX_SKB_FRAGS, probably missing something). cxgb3 may be able to do 64kB
DMAs (again not sure), but you could imagine a card with DMA restrictions,
perhaps as bad as 4kB per descriptor. In this case MAX_SKB_FRAGS is sized way
too small.

Thinking out aloud, would setting a pessimistic value for MAX_SKB_FRAGS
be one way to fix this? ie:

#define MAX_SKB_FRAGS (65536/4096 + 2)

I'm not sure if that would break the tx side of some adapters.

Anton

View attachment "ixgbe_hw_gro.patch" of type "text/x-diff" (1770 bytes)