netdev - Re: Scalability of interface creation and deletion

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <443ACEB21CD1E406E4AE377F@nimrod.local>
Date:	Sun, 08 May 2011 11:09:35 +0100
From:	Alex Bligh <alex@...x.org.uk>
To:	Eric Dumazet <eric.dumazet@...il.com>
cc:	netdev@...r.kernel.org, Alex Bligh <alex@...x.org.uk>
Subject: Re: Scalability of interface creation and deletion

--On 7 May 2011 20:39:13 +0200 Eric Dumazet <eric.dumazet@...il.com> wrote:

> Le samedi 07 mai 2011 à 20:32 +0200, Eric Dumazet a écrit :
>
> Also you could patch synchronize_sched() itself instead of
> synchronize_net()

OK, I did this, plus instrumented the call to rcu_barrier()
you mentioned:

Looking at the synchronize_net() and rcu_barrier() calls:
 Total 8.43935 Usage 399 Average 0.02115 elsewhere
 Total 10.65050 Usage 200 Average 0.05325 rcu_barrier
 Total 9.28948 Usage 200 Average 0.04645 synchronize_net

it's spending about 1/3 of its time in that rcu_barrier, 1/3
in synchronize_sched() and 1/3 elsewere.

Turning now to the synchronize_sched() (per your patch), I see

  Total 16.36852 Usage 400 Average 0.04092 synchronize_sched()

Note "Usage 400". That's because precisely half the calls to
synchronize_sched() occur outside of synchronize_net(), and
half occur within synchronize_net() (per logs)

A typical interface being removed looks like this:
May  8 09:47:31 nattytest kernel: [  177.030197] synchronize_sched() in 
66921 us
May  8 09:47:31 nattytest kernel: [  177.030957] begin synchronize_net()
May  8 09:47:31 nattytest kernel: [  177.120085] synchronize_sched() in 
89080 us
May  8 09:47:31 nattytest kernel: [  177.120819] end synchronize_net()
May  8 09:47:31 nattytest kernel: [  177.121698] begin rcu_barrier()
May  8 09:47:31 nattytest kernel: [  177.190152] end rcu_barrier()

So for every interface being destroyed (I'm doing 200 as veths
are pairs), we do 2 synchronize_sched() calls and 1 rcu_barrier.
Each of these takes roughly 42ms with CONFIG_HZ set to 100,
leading to 125ms per interface destroy, and 250ms per veth
pair destroy.

It may be a naive question but why would we need to do
2 synchronize_sched() and 1 rcu_barrier() to remove an
interface?

-- 
Alex Bligh
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html