netdev - Re: [RFC Patch net-next] net_sched: make classifying lockless on ingress

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <52B924F1.9020201@gmail.com>
Date:	Mon, 23 Dec 2013 22:08:49 -0800
From:	John Fastabend <john.fastabend@...il.com>
To:	Cong Wang <xiyou.wangcong@...il.com>
CC:	Jamal Hadi Salim <jhs@...atatu.com>,
	John Fastabend <john.r.fastabend@...el.com>,
	Eric Dumazet <eric.dumazet@...il.com>,
	Linux Kernel Network Developers <netdev@...r.kernel.org>,
	"David S. Miller" <davem@...emloft.net>
Subject: Re: [RFC Patch net-next] net_sched: make classifying lockless on
 ingress

On 12/23/2013 04:56 PM, Cong Wang wrote:
> On Sat, Dec 21, 2013 at 3:09 PM, John Fastabend
> <john.fastabend@...il.com> wrote:
>>
>> I solved this by making them per CPU and synchronizing when I hit
>> an operation that required sync'ing them. Going forward if folks
>> have the time to write SMP aware qdisc's that work with eventually
>> consistent counters that would be great.
>>
>
> Interesting, then you have to copy the same filters and actions
> to all per-cpu-ingress-qdisc, right? Also you need to handle
> CPU online/offline event.
>
> The number of CPU's grows fast today, so the total size
> of such ingress qdisc would be huge if I install lots
> of filters and action.
>

In this case I was specifically talking about statistics so the
bstats and qstats.

As long as the qdisc's do not require global state this works well
enough. However as Jamal keeps pointing out the problem is any qdisc
which requires global state requires locking (I paraphrase but I think
replicate the spirit correctly) and this doesn't work well with many
CPUs. So you either replicate the qdiscs one per queue like we do in
the mq and mqprio case effectively removing any global state or you
develop qdiscs that don't require global state or at least work with
eventually consistent data to avoid the constant syncing of data.

I think though a qdisc per nic queue is really not as bad as you think.
For example we do this on the tx side and it works OK. Note its per
RX queue and not per CPU.

.John

-- 
John Fastabend         Intel Corporation
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html