netdev - why all packets have same queue no when rps enabled?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 21 Feb 2011 20:07:49 -0800
From:	Jon Zhou <Jon.Zhou@...u.com>
To:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: why all packets have same queue no when rps enabled?

Hi

I expect each incoming packet will have a different queue no. when I enabled RPS on kernel 2.6.36.4

cat /sys/class/net/eth4/queues/rx-0/rps_cpus
00000000,000000ff

CPU0       CPU1       CPU2       CPU3       CPU4       CPU5       CPU6       CPU7       CPU8       CPU9       CPU10      CPU11      CPU12      CPU13      CPU14      CPU15      CPU16
      CPU17      CPU18      CPU19      CPU20      CPU21      CPU22      CPU23      CPU24      CPU25      CPU26      CPU27      CPU28      CPU29      CPU30      CPU31
     HI:          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0          0
         0          0          0          0          0          0          0          0          0          0          0          0          0          0          0
   TIMER:    6027512     710165    2623243     542768     427807     217424     192940     217043          0          0          0          0          0          0          0          0          0
         0          0          0          0          0          0          0          0          0          0          0          0          0          0          0
  NET_TX:    1365741         59     750957          0        171          0          3          0          0          0          0          0          0          0          0          0          0
         0          0          0          0          0          0          0          0          0          0          0          0          0          0          0
  NET_RX:   40465750   11140803    8545859   14417762    8913471   12298691   14216845    3348431   < ---- indeed spread across cpus


I manually disable RSS on the intel X520 multiqueue supported NIC.
Cat /proc/interrupts

  87:   21348294          0          0          0          0          0          0          0   PCI-MSI-edge      eth4-rx-0
  88:      38394          0          0          0          0          0          0          0   PCI-MSI-edge      eth4-tx-0



When I tried the below program to filter packet by queue no.I got these results:

struct sock_filter BPF_code[]= {     
    { 0x28,0,0,SKF_AD_OFF+SKF_AD_QUEUE},
    { 0x15, 0, 1, 0x00000001 },
    { 0x6, 0, 0, 0x0000ffff },
    { 0x6, 0, 0, 0x00000000 }
  };

  struct sock_fprog Filter; 
    
  Filter.len = 4;//15;
  Filter.filter = BPF_code;
  
  if ( (sock=socket(PF_PACKET, SOCK_RAW, htons(ETH_P_IP)))<0) {
    perror("socket");
    exit(1);
  }

  /* Set the network card in promiscuos mode */
  strncpy(ethreq.ifr_name,"eth4",IFNAMSIZ);
  if (ioctl(sock,SIOCGIFFLAGS,&ethreq)==-1) {
    perror("ioctl");
    close(sock);
    exit(1);
  }
  ethreq.ifr_flags|=IFF_PROMISC;
  if (ioctl(sock,SIOCSIFFLAGS,&ethreq)==-1) {
    perror("ioctl");
    close(sock);
    exit(1);
  }

  /* Attach the filter to the socket */
  if(setsockopt(sock, SOL_SOCKET, SO_ATTACH_FILTER,&Filter, sizeof(Filter))<0){
    perror("setsockopt");
    close(sock);
    exit(1);
  }
  static int count = 0;
  while (1) {
    printf("#%d----------\n",count++);
    n = recvfrom(sock,buffer,2048,0,NULL,NULL);
    printf("%d bytes read\n",n);
...
}


Looks almost all packets fall at same queue?
Will RPS allocate queue no for each packet? and what hash algorithm rps used? (is it Toeplitz hash algorithm?)

jon
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html