lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Thu, 24 May 2012 17:09:53 +0000
From:	Arvid Brodin <Arvid.Brodin@...n.com>
To:	Stephen Hemminger <shemminger@...tta.com>
CC:	"netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"David S. Miller" <davem@...emloft.net>,
	Bruno Ferreira <balferreira@...glemail.com>,
	Christian Borntraeger <borntraeger@...ibm.com>,
	Herbert Xu <herbert@...dor.apana.org.au>
Subject: Re: [RFC] net/hsr: Add support for IEC 62439-3 High-availability
 Seamless Redundancy

On 2012-05-14 20:28, Stephen Hemminger wrote:
> On Mon, 14 May 2012 18:11:44 +0000
> Arvid Brodin <Arvid.Brodin@...n.com> wrote:
> 
>> On 2012-03-27 15:20, Arvid Brodin wrote:
>>> Hi!
>>
>> *snip*
>>>
>>> 2) I have a locking problem that I haven't managed to figure out. This happens
>>>    the first time I send any packet (hsr_dev_xmit() in hsr_device.c:121, called
>>>    from hsr_device.c:147). It happens even if I set skb2 to NULL (i.e. only send
>>>    one copy):
>>>
>>> =============================================
>>> [ INFO: possible recursive locking detected ]
>>> 2.6.37 #118
>>> ---------------------------------------------
>>> swapper/0 is trying to acquire lock:
>>>  (_xmit_ETHER#2){+.-...}, at: [<901bf38e>] sch_direct_xmit+0x24/0x152
>>>
>>> but task is already holding lock:
>>>  (_xmit_ETHER#2){+.-...}, at: [<901b4d1a>] dev_queue_xmit+0x31e/0x3cc
>>>
>>> other info that might help us debug this:
>>> 4 locks held by swapper/0:
>>>  #0:  (&n->timer){+.-...}, at: [<9002bc20>] run_timer_softirq+0x98/0x184
>>>  #1:  (rcu_read_lock_bh){.+....}, at: [<901b49fc>] dev_queue_xmit+0x0/0x3cc
>>>  #2:  (_xmit_ETHER#2){+.-...}, at: [<901b4d1a>] dev_queue_xmit+0x31e/0x3cc
>>>  #3:  (rcu_read_lock_bh){.+....}, at: [<901b49fc>] dev_queue_xmit+0x0/0x3cc
>>>
>>> stack backtrace:
>>> Call trace:
>>>  [<9001c640>] dump_stack+0x18/0x20
>>>  [<90040eac>] validate_chain+0x40c/0x9ac
>>>  [<90041a58>] __lock_acquire+0x60c/0x670
>>>  [<90042f32>] lock_acquire+0x3a/0x48
>>>  [<902201a4>] _raw_spin_lock+0x20/0x44
>>>  [<901bf38e>] sch_direct_xmit+0x24/0x152
>>>  [<901b4c14>] dev_queue_xmit+0x218/0x3cc
>>>  [<9021c2e0>] slave_xmit+0x10/0x14
>>>  [<9021c540>] hsr_dev_xmit+0x88/0x8c
>>>  [<901b4942>] dev_hard_start_xmit+0x3c6/0x480
>>>  [<901b4d34>] dev_queue_xmit+0x338/0x3cc
>>>  [<901e3cd8>] arp_xmit+0x8/0xc
>>>  [<901e4436>] arp_send+0x2a/0x2c
>>>  [<901e4e74>] arp_solicit+0x15c/0x170
>>>  [<901bad0c>] neigh_timer_handler+0x1c0/0x204
>>>  [<9002bc8a>] run_timer_softirq+0x102/0x184
>>>  [<900287d8>] __do_softirq+0x64/0xe0
>>>  [<9002896a>] do_softirq+0x26/0x48
>>>  [<90028a66>] irq_exit+0x2e/0x64
>>>  [<90019f16>] do_IRQ+0x46/0x5c
>>>  [<90018428>] irq_level0+0x18/0x60
>>>  [<9021cc16>] rest_init+0x72/0x98
>>>  [<9000063c>] start_kernel+0x21c/0x258
>>>  [<00000000>] 0x0
>>>
>>>    Any idea why this happens? I need help!
>>
>>
>> I've spent a few days digging into this and the key apparently is NETIF_F_LLTX.
>>
>> The problem seems to be that HARD_TX_LOCK is called more than once, first for my virtual
>> hsr device and then, recursively, for each of the slaves in turn. (At least that's where
>> lockdep complains - at __netif_tx_lock(), that is.)
>>
>> At first I just could not understand why both the VLAN and the bonding code got away with
>> recursive calls to dev_queue_xmit() but I didn't. After some gooling (a lot, actually) I
>> found some references to the NETIF_F_LLTX flag (here's one:
>> http://lwn.net/Articles/121566/). I realised both VLAN and bonding code set this flag. And
>> sure enough, if I set it for my hsr device lockdep does not complain any more.
>>
>> But NETIF_F_LLTX is described as deprecated in both netdevice.h and in
>> Documentation/networking/netdevices.txt. Is there an alternative solution that I should
>> use instead?
>>
>> (To recap, a hsr device is a virtual device which uses two Ethernet devices as slaves.
>> This gives redundancy with instant failover, and since nodes are connected in a ring
>> topology, uses less cabling than duplication.)
>>
> 
> LLTX is deprecated (ie should not be used) for physical devices.
> 
> Also, for virtual devices, there should be non transmit queue, this
> causes mulit-queue lockless semantics to be preserved as the call passes
> through the virtual device.

(First: apologies for my late reply; your emails doesn't get through to me for some reason.)

So does this mean that it is OK to use LLTX for virtual devices? My virtual device has
zero queue length (no transmit queue), but since it calls dev_queue_xmit for its slaves, I
still get recursive locking if I don't set LLTX (just like vlan and bonding does).

-- 
Arvid Brodin | Consultant (Linux)
XDIN AB | Jan Stenbecks Torg 17 | SE-164 40 Kista | Sweden | xdin.com--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists