| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <1298355182.3360.9.camel@edumazet-laptop>
Date: Tue, 22 Feb 2011 07:13:02 +0100
From: Eric Dumazet <eric.dumazet@...il.com>
To: Jon Zhou <Jon.Zhou@...u.com>
Cc: "netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: RE: why all packets have same queue no when rps enabled?
Le lundi 21 février 2011 à 21:56 -0800, Jon Zhou a écrit :
>
> > -----Original Message-----
> > From: Eric Dumazet [mailto:eric.dumazet@...il.com]
> > Sent: Tuesday, February 22, 2011 1:41 PM
> > To: Jon Zhou
> > Cc: netdev@...r.kernel.org
> > Subject: Re: why all packets have same queue no when rps enabled?
> >
> > Le lundi 21 février 2011 à 20:07 -0800, Jon Zhou a écrit :
> > > Hi
> > >
> > > I expect each incoming packet will have a different queue no. when I
> > enabled RPS on kernel 2.6.36.4
> > >
> > > cat /sys/class/net/eth4/queues/rx-0/rps_cpus
> > > 00000000,000000ff
> > >
> > > CPU0 CPU1 CPU2 CPU3 CPU4 CPU5
> > CPU6 CPU7 CPU8 CPU9 CPU10 CPU11 CPU12
> > CPU13 CPU14 CPU15 CPU16
> > > CPU17 CPU18 CPU19 CPU20 CPU21 CPU22
> > CPU23 CPU24 CPU25 CPU26 CPU27 CPU28 CPU29
> > CPU30 CPU31
> > > HI: 0 0 0 0 0
> > 0 0 0 0 0 0 0
> > 0 0 0 0 0
> > > 0 0 0 0 0 0
> > 0 0 0 0 0 0 0
> > 0 0
> > > TIMER: 6027512 710165 2623243 542768 427807
> > 217424 192940 217043 0 0 0
> > 0 0 0 0 0 0
> > > 0 0 0 0 0 0
> > 0 0 0 0 0 0 0
> > 0 0
> > > NET_TX: 1365741 59 750957 0 171
> > 0 3 0 0 0 0 0
> > 0 0 0 0 0
> > > 0 0 0 0 0 0
> > 0 0 0 0 0 0 0
> > 0 0
> > > NET_RX: 40465750 11140803 8545859 14417762 8913471
> > 12298691 14216845 3348431 < ---- indeed spread across cpus
> > >
> > >
> > > I manually disable RSS on the intel X520 multiqueue supported NIC.
> > > Cat /proc/interrupts
> > >
> > > 87: 21348294 0 0 0 0
> > 0 0 0 PCI-MSI-edge eth4-rx-0
> > > 88: 38394 0 0 0 0
> > 0 0 0 PCI-MSI-edge eth4-tx-0
> > >
> > >
> > >
> > > When I tried the below program to filter packet by queue no.I got
> > these results:
> > >
> > > struct sock_filter BPF_code[]= {
> > > { 0x28,0,0,SKF_AD_OFF+SKF_AD_QUEUE},
> > > { 0x15, 0, 1, 0x00000001 },
> > > { 0x6, 0, 0, 0x0000ffff },
> > > { 0x6, 0, 0, 0x00000000 }
> > > };
> > >
> > > struct sock_fprog Filter;
> > >
> > > Filter.len = 4;//15;
> > > Filter.filter = BPF_code;
> > >
> > > if ( (sock=socket(PF_PACKET, SOCK_RAW, htons(ETH_P_IP)))<0) {
> > > perror("socket");
> > > exit(1);
> > > }
> > >
> > > /* Set the network card in promiscuos mode */
> > > strncpy(ethreq.ifr_name,"eth4",IFNAMSIZ);
> > > if (ioctl(sock,SIOCGIFFLAGS,ðreq)==-1) {
> > > perror("ioctl");
> > > close(sock);
> > > exit(1);
> > > }
> > > ethreq.ifr_flags|=IFF_PROMISC;
> > > if (ioctl(sock,SIOCSIFFLAGS,ðreq)==-1) {
> > > perror("ioctl");
> > > close(sock);
> > > exit(1);
> > > }
> > >
> > > /* Attach the filter to the socket */
> > > if(setsockopt(sock, SOL_SOCKET, SO_ATTACH_FILTER,&Filter,
> > sizeof(Filter))<0){
> > > perror("setsockopt");
> > > close(sock);
> > > exit(1);
> > > }
> > > static int count = 0;
> > > while (1) {
> > > printf("#%d----------\n",count++);
> > > n = recvfrom(sock,buffer,2048,0,NULL,NULL);
> > > printf("%d bytes read\n",n);
> > > ...
> > > }
> > >
> > >
> > > Looks almost all packets fall at same queue?
> > > Will RPS allocate queue no for each packet? and what hash algorithm
> > rps used? (is it Toeplitz hash algorithm?)
> > >
> >
> > I believe you are mistaken.
> >
> > RPS is not there to spread load on _all_ cpus, but to use a hash
> > function so that all packets of a given flow are directed to a given
> > cpu.
> >
> > If you receive 1.000.000 packets of the same flow, they all are
> > delivered to one CPU.
> >
> >
> With RSS supported NIC, I saw packet#0,1,2,3,~packet#7 will be delivered to different queue, but after RSS disabled and RPS turn on, all these packet will be allocated same queue no(I used the above filter program to find that)
> Is there any fault in the filter program?
>
I dont know...
The easy way to find out is to actually do "cat /proc/net/softnet_stat"
Since the last column is the number of time a cpu was given a packet by
another cpu (RPS or RFS action)
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists