netdev - RE: [patch iproute2 v6 0/3] tc: Add -bs option to batch mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <VI1PR0501MB2143DEEBE30BA02E4ED129C0AB130@VI1PR0501MB2143.eurprd05.prod.outlook.com>
Date:   Mon, 8 Jan 2018 02:03:53 +0000
From:   Chris Mi <chrism@...lanox.com>
To:     Phil Sutter <phil@....cc>, "dsahern@...il.com" <dsahern@...il.com>,
        "marcelo.leitner@...il.com" <marcelo.leitner@...il.com>
CC:     "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "gerlitz.or@...il.com" <gerlitz.or@...il.com>,
        "stephen@...workplumber.org" <stephen@...workplumber.org>
Subject: RE: [patch iproute2 v6 0/3] tc: Add -bs option to batch mode

> -----Original Message-----
> From: n0-1@...yte.nwl.cc [mailto:n0-1@...yte.nwl.cc] On Behalf Of Phil
> Sutter
> Sent: Saturday, January 6, 2018 1:25 AM
> To: Chris Mi <chrism@...lanox.com>
> Cc: netdev@...r.kernel.org; gerlitz.or@...il.com;
> stephen@...workplumber.org; dsahern@...il.com;
> marcelo.leitner@...il.com
> Subject: Re: [patch iproute2 v6 0/3] tc: Add -bs option to batch mode
> 
> Hi Chris,
> 
> On Thu, Jan 04, 2018 at 04:34:51PM +0900, Chris Mi wrote:
> > Currently in tc batch mode, only one command is read from the batch
> > file and sent to kernel to process. With this patchset, we can
> > accumulate several commands before sending to kernel. The batch size
> > is specified using option -bs or -batchsize.
> >
> > To accumulate the commands in tc, client should allocate an array of
> > struct iovec. If batchsize is bigger than 1, only after the client has
> > accumulated enough commands, can the client call rtnl_talk_msg to send
> > the message that includes the iov array. One exception is that there
> > is no more command in the batch file.
> >
> > But please note that kernel still processes the requests one by one.
> > To process the requests in parallel in kernel is another effort.
> > The time we're saving in this patchset is the user mode and kernel
> > mode context switch. So this patchset works on top of the current kernel.
> >
> > Using the following script in kernel, we can generate 1,000,000 rules.
> > 	tools/testing/selftests/tc-testing/tdc_batch.py
> >
> > Without this patchset, 'tc -b $file' exection time is:
> >
> > real    0m15.555s
> > user    0m7.211s
> > sys     0m8.284s
> >
> > With this patchset, 'tc -b $file -bs 10' exection time is:
> >
> > real    0m13.043s
> > user    0m6.479s
> > sys     0m6.504s
> >
> > The insertion rate is improved more than 10%.
> 
> Did you measure the effect of increasing batch sizes?
Yes. Even if we enlarge the batch size bigger than 10, there is no big improvement.
I think that's because current kernel doesn't process the requests in parallel.
If kernel processes the requests in parallel, I believe specifying a bigger batch size
will get a better result.
> 
> I wonder whether specifying the batch size is necessary at all. Couldn't batch
> mode just collect messages until either EOF or an incompatible command is
> encountered which then triggers a commit to kernel? This might simplify
> code quite a bit.
That's a good suggestion.

-Chris
> 
> Cheers, Phil