netdev - Re: root_lock vs. device's TX lock

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1321550786.2751.83.camel@edumazet-HP-Compaq-6005-Pro-SFF-PC>
Date:	Thu, 17 Nov 2011 18:26:26 +0100
From:	Eric Dumazet <eric.dumazet@...il.com>
To:	Tom Herbert <therbert@...gle.com>
Cc:	Linux Netdev List <netdev@...r.kernel.org>
Subject: Re: root_lock vs. device's TX lock

Le jeudi 17 novembre 2011 à 17:34 +0100, Eric Dumazet a écrit :
> Le jeudi 17 novembre 2011 à 08:10 -0800, Tom Herbert a écrit :
> > From sch_direct_xmit:
> > 
> >         /* And release qdisc */
> >         spin_unlock(root_lock);
> > 
> >         HARD_TX_LOCK(dev, txq, smp_processor_id());
> >         if (!netif_tx_queue_frozen_or_stopped(txq))
> >                 ret = dev_hard_start_xmit(skb, dev, txq);
> > 
> >         HARD_TX_UNLOCK(dev, txq);
> > 
> >         spin_lock(root_lock);
> > 
> > This is a lot of lock manipulation to basically switch from one lock
> > to another and possibly thrashing just to send a packet.  I am
> > thinking that if the there is a 1-1 correspondence between qdisc and
> > device queue then we could actually use the device's lock as the root
> > lock for the qdisc.  So in that case, we would need to touch any locks
> > from sch_direct_xmit (just hold root lock which is already device lock
> > for the duration).
> > 
> > Is there any reason why this couldn't work?
> 
> But we have to dirty part of Qdisc anyway ?
> (state, bstats, q, ...)
> 

Also we want to permit other cpus to enqueue packets to Qdisc while our
cpu is busy in network driver ndo_start_xmit()

For complex Qdisc / tc setups (potentially touching a lot of cache
lines), we could eventually add a small ring buffer so that the cpu
doing the ndo_start_xmit() also queues the packets into Qdisc.

This ringbuffer could use a lockless algo. (we currently use the
secondary 'busylock' to serialize other cpus, but each cpu calls qdisc
enqueue itself.)

--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html