netdev - Re: RFS configuration questions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 02 Dec 2010 22:40:41 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Shawn Bohrer <sbohrer@...advisors.com>
Cc:	netdev@...r.kernel.org, therbert@...gle.com
Subject: Re: RFS configuration questions

Le jeudi 02 décembre 2010 à 15:16 -0600, Shawn Bohrer a écrit :
> I've been playing around with RPS/RFS on my multiqueue 10g Chelsio NIC
> and I've got some questions about configuring RFS.
> 
> I've enabled RPS with:
> 
> for x in $(seq 0 7); do
>     echo FFFFFFFF,FFFFFFFF > /sys/class/net/vlan816/queues/rx-${x}/rps_cpus
> done
> 
> This appears to work when I watch 'mpstat -P ALL 1' as I can see the
> softirq load is now getting distributed across all of the CPUs instead
> of just the four (the card is a two port card and assigns four queues
> per port) original hw receive queues which I have bound to CPUs
> 0-3.
> 
> To enable RFS I've run:
> 
> echo 16384 > /proc/sys/net/core/rps_sock_flow_entries
> 
> Is there any explanation of what this sysctl actually does?  Is this
> the max number of sockets/flows that the kernel can steer?  Is this a
> system wide max, a per interface max, or a per receive queue max?
> 

Yes, some doc is missing...

Its a system wide and shared limit.

> Next I ran:
> 
> for x in $(seq 0 7); do
>     echo 16384 > /sys/class/net/vlan816/queues/rx-${x}/rps_flow_cnt
> done
> 
> Is this correct?  Is these the max number of sockets/flows that can be
> steered per receive queue?  Does the sum of these values need to add
> up to rps_sock_flow_entries (I also tried 2048)? Is this all that is
> needed to enable RFS?
> 

Yes thats all.

> With these settings I can watch 'mpstat -P ALL 1' and it doesn't
> appear RFS has changed the softirq load.  To get a better idea if it
> was working I used taskset to bind my receiving processes to a set of
> cores, yet mpstat still shows the softirq load getting distributed
> across all cores, not just the ones where my receiving processes are
> bound.  Is there a better way to determine if RFS is actually working?
> Have I configured RFS incorrectly?

It seems fine to me, but what kind of workload do you have, and what
version of kernel do you run ?



--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html