netdev - Re: Kernel 4.19 network performance - forwarding/routing normal users traffic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 2 Nov 2018 13:23:56 +0800
From:   Aaron Lu <aaron.lu@...el.com>
To:     Saeed Mahameed <saeedm@...lanox.com>
Cc:     "brouer@...hat.com" <brouer@...hat.com>,
        "pstaszewski@...are.pl" <pstaszewski@...are.pl>,
        "eric.dumazet@...il.com" <eric.dumazet@...il.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Tariq Toukan <tariqt@...lanox.com>,
        "ilias.apalodimas@...aro.org" <ilias.apalodimas@...aro.org>,
        "yoel@...knet.dk" <yoel@...knet.dk>,
        "mgorman@...hsingularity.net" <mgorman@...hsingularity.net>
Subject: Re: Kernel 4.19 network performance - forwarding/routing normal
 users traffic

On Thu, Nov 01, 2018 at 08:23:19PM +0000, Saeed Mahameed wrote:
> On Thu, 2018-11-01 at 23:27 +0800, Aaron Lu wrote:
> > On Thu, Nov 01, 2018 at 10:22:13AM +0100, Jesper Dangaard Brouer
> > wrote:
> > ... ...
> > > Section copied out:
> > > 
> > >   mlx5e_poll_tx_cq
> > >   |          
> > >    --16.34%--napi_consume_skb
> > >              |          
> > >              |--12.65%--__free_pages_ok
> > >              |          |          
> > >              |           --11.86%--free_one_page
> > >              |                     |          
> > >              |                     |--10.10%
> > > --queued_spin_lock_slowpath
> > >              |                     |          
> > >              |                      --0.65%--_raw_spin_lock
> > 
> > This callchain looks like it is freeing higher order pages than order
> > 0:
> > __free_pages_ok is only called for pages whose order are bigger than
> > 0.
> 
> mlx5 rx uses only order 0 pages, so i don't know where these high order
> tx SKBs are coming from.. 

Perhaps here:
__netdev_alloc_skb(), __napi_alloc_skb(), __netdev_alloc_frag() and
__napi_alloc_frag() will all call page_frag_alloc(), which will use
__page_frag_cache_refill() to get an order 3 page if possible, or fall
back to an order 0 page if order 3 page is not available.

I'm not sure if your workload will use the above code path though.