netdev - Re: [RFC] bnx2x: Insane RX rings

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4C8953D3.9060204@hp.com>
Date:	Thu, 09 Sep 2010 14:38:27 -0700
From:	Rick Jones <rick.jones2@...com>
To:	David Miller <davem@...emloft.net>
CC:	ole@....pl, eric.dumazet@...il.com, netdev@...r.kernel.org,
	eilong@...adcom.com
Subject: Re: [RFC] bnx2x: Insane RX rings

David Miller wrote:
> From: Krzysztof Olędzki <ole@....pl>
> Date: Thu, 09 Sep 2010 23:21:01 +0200
> 
> 
>>On 2010-09-09 22:45, Eric Dumazet wrote:
>>
>>>Problem is : With 16 RX queues per device , thats 4078*16*2Kbytes per
>>>ethernet port.
>>>
>>>Total :
>>>
>>>skbuff_head_cache 130747 131025 256 15 1 : tunables 120 60 8 :
>>>slabdata 8735 8735 40
>>>size-2048 130866 130888 2048 2 1 : tunables 24 12 8 : slabdata 65444
>>>65444 28
>>>
>>>Thats about 300 Mbytes of memory, just in case some network trafic
>>>will occur.
>>>
>>>Lets do something about that ?
>>
>>Yep, it is ~8MB per queue, not so much alone, but a lot together. For
>>this reason I use something like bnx2.num_queues=2 on servers where I
>>don't need much CPU power for network workload.
> 
> 
> I think simply that the RX queue size should be scaled by the number
> of queues we have.
> 
> If people want enormous RX ring sizes even when there are many queues,
> they can use ethtool to get that.
> 
> Taking up 130MB of memory per-card, just for RX packet buffers, is
> certainly over the top.

It gets even better if one consideres JumboFrames...  that said, I've had 
customer contacts (indirect) where they were quite keep to have a ring size of 
at least 2048 packets - I never could get it confirmed, but I suspect they had 
applications/systems that might "go out to lunch" for long-enough periods of 
time they wanted that degree of FIFO.

Doesn't necessarily change "what should be the defaults" much but there it is.

rick jones
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html