lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Wed, 18 Mar 2009 14:07:32 -0700 From: Vernon Mauery <vernux@...ibm.com> To: Andi Kleen <andi@...stfloor.org> CC: netdev <netdev@...r.kernel.org>, LKML <linux-kernel@...r.kernel.org>, rt-users <linux-rt-users@...r.kernel.org> Subject: Re: High contention on the sk_buff_head.lock Andi Kleen wrote: > Vernon Mauery <vernux@...ibm.com> writes: >> So while this issue really hits -rt kernels hard, it has a real effect on >> mainline kernels as well. The contention of the spinlocks is amplified >> when they get turned into rt-mutexes, which causes a double context switch. > > The new adaptive spin heuristics that have been discussed some time > ago didn't help? Unsurprisingly making locks a lot more expensive > has drawbacks as you discovered. Yes. Well, while the adaptive spinlocks did great things for the network throughput last time I tested them, they also didn't quite give the determinism in other areas. It would be nice to be able to target a handful of trouble locks with adaptive spinlocks. Even so, though I saw dramatic throughput increases with adaptive spinlocks, they would still be bound by this same lock contention that I am seeing when the locks are true spinlocks. >> &list->lock#3: 24517307 24643791 0.71 1286.62 56516392.42 34834296 44904018 0.60 164.79 31314786.02 >> ------------- >> &list->lock#3 15596927 [<ffffffff812474da>] dev_queue_xmit+0x2ea/0x468 > > The real "fix" would be probably to use a multi queue capable NIC > and a NIC driver that sets up multiple queues for TX (normally they > only do for RX). Then cores or a set of cores (often the number > of cores is larger than the number of NIC queues) could avoid this > problem. Disadvantage: more memory use. Hmmm. So do either the netxen_nic or bnx2x drivers support multiple queues? (that is the HW that I have access to right now). And do I need to do anything to set them up? > But then again I'm not sure it's worth it if the problem only > happens in out of tree RT. The effects of the high contention are not quite so pronounced in the vanilla kernel, but I think we are still limited by this lock. In the -rt kernel, it is obvious that the lock contention is causing lots of trouble. --Vernon -- To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists