| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 9 Mar 2012 17:54:48 +0000 From: Dave Taht <dave.taht@...il.com> To: "John A. Sullivan III" <jsullivan@...nsourcedevel.com> Cc: John Fastabend <john.r.fastabend@...el.com>, netdev@...r.kernel.org Subject: Re: Requeues On Sat, Mar 3, 2012 at 10:13 AM, John A. Sullivan III <jsullivan@...nsourcedevel.com> wrote: > On Fri, 2012-03-02 at 22:54 -0800, John Fastabend wrote: >> On 3/2/2012 9:48 PM, John A. Sullivan III wrote: >> > Hello, all. I am seeing a small but significant number of requeues on >> > our pfifo_fast qdiscs. I've not been able to find much on what this >> > means but the little I have implies it may be a problem with the >> > physical interfaces. However, these are fairly high end systems with >> > Intel e1000 quad port cards. >From what you describe it sounds like this is not a 'problem', but a sign that your environment is capable of saturating e1000 quad port cards. Adding a little active queue management (red or sfqred) will reduce the impact of the extra buffering and lead to more responsive streams with more accurate estimates of the BDP. if you are already running bql, you will generally see more requeues. >> > >> > We thought it might have something to do with the bonded interfaces so >> > we checked some other high end systems without bonded interfaces but the >> > same quad port cards and, lo and behold, the same small but significant >> > number of requeues. >> > >> > Is this normal or does it indicate a problem somewhere? Thanks - John >> > >> > -- >> > To unsubscribe from this list: send the line "unsubscribe netdev" in >> > the body of a message to majordomo@...r.kernel.org >> > More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> One of two things can happen to cause the requeue counter to increment. >> >> When the qdisc dequeues a packet it then gets enqueued in yet another >> queue in the e1000 driver. If the qdisc is dequeueing packets faster >> than the hardware can consume them the driver will return a >> NETDEV_TX_BUSY error. This causes the qdisc to 'requeue' the packet and >> in the process increments this counter. >> >> The gist is pfifo_fast dequeued a packet tried to give it to e1000 which >> pushed back so pfifo_fast put the packet back on the queue. >> >> By the way you can tune the size of the e1000 queues manually by playing >> with: >> >> /sys/class/net/ethx/tx_queue_len >> >> But you likely don't want to make them any larger else you'll hit the >> buffer bloat problem. I'm guessing we should add byte queue limit support >> here. >> >> The second way to trigger this is multiple cpus contending for a lock. >> >> In short requeue counts as long as its not excessive are just part >> of normal operation. So shouldn't be a problem. > <snip> > Thank you very much. I was not aware of CPU contention being a cause. > We did notice the problem primarily on systems with 8 to 16 cores - John > > -- > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to majordomo@...r.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Dave Täht SKYPE: davetaht US Tel: 1-239-829-5608 http://www.bufferbloat.net -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists