netdev - Re: root_lock vs. device's TX lock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA93jw6VfGKAk_gBZOjyCCcR0tCSc8PMjOoPv6Qi8m+P1idp4Q@mail.gmail.com>
Date:	Thu, 17 Nov 2011 19:19:58 +0100
From:	Dave Taht <dave.taht@...il.com>
To:	Eric Dumazet <eric.dumazet@...il.com>
Cc:	Tom Herbert <therbert@...gle.com>,
	Linux Netdev List <netdev@...r.kernel.org>
Subject: Re: root_lock vs. device's TX lock

On Thu, Nov 17, 2011 at 6:26 PM, Eric Dumazet <eric.dumazet@...il.com> wrote:
> Le jeudi 17 novembre 2011 à 17:34 +0100, Eric Dumazet a écrit :
>> Le jeudi 17 novembre 2011 à 08:10 -0800, Tom Herbert a écrit :
>> > From sch_direct_xmit:
>> >
>> >         /* And release qdisc */
>> >         spin_unlock(root_lock);
>> >
>> >         HARD_TX_LOCK(dev, txq, smp_processor_id());
>> >         if (!netif_tx_queue_frozen_or_stopped(txq))
>> >                 ret = dev_hard_start_xmit(skb, dev, txq);
>> >
>> >         HARD_TX_UNLOCK(dev, txq);
>> >
>> >         spin_lock(root_lock);
>> >
>> > This is a lot of lock manipulation to basically switch from one lock
>> > to another and possibly thrashing just to send a packet.  I am
>> > thinking that if the there is a 1-1 correspondence between qdisc and
>> > device queue then we could actually use the device's lock as the root
>> > lock for the qdisc.  So in that case, we would need to touch any locks
>> > from sch_direct_xmit (just hold root lock which is already device lock
>> > for the duration).
>> >
>> > Is there any reason why this couldn't work?
>>
>> But we have to dirty part of Qdisc anyway ?
>> (state, bstats, q, ...)
>>
>
> Also we want to permit other cpus to enqueue packets to Qdisc while our
> cpu is busy in network driver ndo_start_xmit()
>
> For complex Qdisc / tc setups (potentially touching a lot of cache
> lines), we could eventually add a small ring buffer so that the cpu
> doing the ndo_start_xmit() also queues the packets into Qdisc.
>
> This ringbuffer could use a lockless algo. (we currently use the
> secondary 'busylock' to serialize other cpus, but each cpu calls qdisc
> enqueue itself.)

I was thinking ringbuffering might also help in adding a 'grouper'
abstraction to the dequeuing side.
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Dave Täht
SKYPE: davetaht
US Tel: 1-239-829-5608
FR Tel: 0638645374
http://www.bufferbloat.net
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html