linux-kernel - Re: [LKP] [tcp] 8b27dae5a2: netperf.Throughput

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20190612035133.GA18313@shbuild999.sh.intel.com>
Date:   Wed, 12 Jun 2019 11:52:29 +0800
From:   Feng Tang <feng.tang@...el.com>
To:     Eric Dumazet <edumazet@...gle.com>
Cc:     kernel test robot <rong.a.chen@...el.com>,
        Stephen Rothwell <sfr@...b.auug.org.au>,
        Willem de Bruijn <willemb@...gle.com>,
        Soheil Hassas Yeganeh <soheil@...gle.com>,
        LKML <linux-kernel@...r.kernel.org>, "lkp@...org" <lkp@...org>,
        "David S. Miller" <davem@...emloft.net>, ying.huang@...el.com
Subject: Re: [LKP] [tcp] 8b27dae5a2: netperf.Throughput_Mbps -25.7% regression

On Tue, Jun 04, 2019 at 06:07:35PM +0800, Feng Tang wrote:
> On Thu, May 30, 2019 at 11:23:14PM +0800, Feng Tang wrote:
> > Hi Eric,
> > 
> > On Thu, May 30, 2019 at 05:21:40AM -0700, Eric Dumazet wrote:
> > > On Thu, May 30, 2019 at 3:31 AM Feng Tang <feng.tang@...el.com> wrote:
> > > >
> > > > On Wed, Apr 03, 2019 at 02:34:36PM +0800, kernel test robot wrote:
> > > > > Greeting,
> > > > >
> > > > > FYI, we noticed a -25.7% regression of netperf.Throughput_Mbps due to commit:
> > > > >
> > > > >
> > > > > commit: 8b27dae5a2e89a61c46c6dbc76c040c0e6d0ed4c ("tcp: add one skb cache for rx")
> > > > > https://git.kernel.org/cgit/linux/kernel/git/next/linux-next.git master
> > > >
> > > > Hi Eric,
> > > >
> > > > Could you help to check this? thanks,
> > > >
> > > 
> > > Hmmm... patch is old and had some bugs that have been fixed.
> > > 
> > > What numbers do you have with more recent kernels ?
> > 
> > 
> > I just run the test  with 5.2-rc2, and the regression is still there.
> 
> Hi Eric,
> 
> Any hint on this?
> 
> >From the perf data, the spinlock contention has an obvious increase:
> 
> 9.28            +7.6       16.91        perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock.free_one_page.__free_pages_ok.___pskb_trim
> 18.55           +8.6       27.14        perf-profile.calltrace.cycles-pp.native_queued_spin_lock_slowpath._raw_spin_lock_irqsave.get_page_from_freelist.__alloc_pages_nodemask.skb_page_frag_refill

Hi Eric,

Any thoughts?

Actually I did some further check. The increased lock contention
comes from the mm zone lock for page alloc/free. I did an 
experiment by changing the SKB_FRAG_PAGE_ORDER from 32K to 64K,
the lock contention is dramatically reduced, and the throughput
got some recovery (10% ~ 15% gain) depending on HW platform, but
can't fully recover the -25.7% loss. Hope this info helps.

Thanks,
Feng

> 
> And for commit 8b27dae5a2 ("tcp: add one skb cache for rx"), IIUC, it
> is not a real cache like the "tx skb cache" patch, and kind of a
> delayed freeing.
> 
> Thanks,
> Feng
> 
>