netdev - [patch iproute2 v6 0/3] tc: Add -bs option to batch mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20180104073454.11867-1-chrism@mellanox.com>
Date:   Thu,  4 Jan 2018 16:34:51 +0900
From:   Chris Mi <chrism@...lanox.com>
To:     netdev@...r.kernel.org
Cc:     gerlitz.or@...il.com, stephen@...workplumber.org,
        dsahern@...il.com, marcelo.leitner@...il.com
Subject: [patch iproute2 v6 0/3] tc: Add -bs option to batch mode

Currently in tc batch mode, only one command is read from the batch
file and sent to kernel to process. With this patchset, we can accumulate
several commands before sending to kernel. The batch size is specified
using option -bs or -batchsize.

To accumulate the commands in tc, client should allocate an array of
struct iovec. If batchsize is bigger than 1, only after the client
has accumulated enough commands, can the client call rtnl_talk_msg
to send the message that includes the iov array. One exception is
that there is no more command in the batch file.

But please note that kernel still processes the requests one by one.
To process the requests in parallel in kernel is another effort.
The time we're saving in this patchset is the user mode and kernel mode
context switch. So this patchset works on top of the current kernel.

Using the following script in kernel, we can generate 1,000,000 rules.
	tools/testing/selftests/tc-testing/tdc_batch.py

Without this patchset, 'tc -b $file' exection time is:

real    0m15.555s
user    0m7.211s
sys     0m8.284s

With this patchset, 'tc -b $file -bs 10' exection time is:

real    0m13.043s
user    0m6.479s
sys     0m6.504s

The insertion rate is improved more than 10%.

In this patchset, we still ack for every rule. If we don't ack at all,
'tc -b $file' exection time is:

real    0m14.748s
user    0m6.944s
sys     0m7.740s

'tc -b $file -bs 10' exection time is:

real    0m12.535s
user    0m6.587s
sys     0m5.888s

We can see that the performance win is to send multiple messages instead
of no acking. I think that's because in tc, we don't spend too much time
processing the ack message.

v3
==
1. Instead of hacking function rtnl_talk directly, add a new function
   rtnl_talk_msg.
2. remove most of global variables to use parameter passing
3. divide the previous patch into 4 patches.

v4
==
1. Remove function setcmdlinetotal. Now in function batch, we read one
   more line to determine if we are reaching the end of file.
2. Remove function __rtnl_check_ack. Now __rtnl_talk calls
__rtnl_talk_msg
   directly.
3. if (batch_size < 1)
        batch_size = 1;

v5
==
1. Fix a bug that can't deal with batch file with blank line.
2. Describe the limitation in man page.

v6
==
1. Add support for mixed commands.
2. Fix a bug that not all messages are acked if batch size > 1.

Chris Mi (3):
  lib/libnetlink: Add a function rtnl_talk_msg
  tc: Add -bs option to batch mode
  man: Add -bs option to tc manpage

 include/libnetlink.h |   3 ++
 lib/libnetlink.c     |  66 +++++++++++++++++++-------
 man/man8/tc.8        |   7 +++
 tc/m_action.c        |  93 +++++++++++++++++++++++++++---------
 tc/tc.c              |  96 ++++++++++++++++++++++++++++++++-----
 tc/tc_common.h       |   8 +++-
 tc/tc_filter.c       | 132 +++++++++++++++++++++++++++++++++++----------------
 7 files changed, 310 insertions(+), 95 deletions(-)

-- 
2.14.3