netdev - Re: Kernel 4.19 network performance - forwarding/routing normal users traffic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9f35c3f5-e865-54db-73bb-960ded60c1cc@itcare.pl>
Date:   Sat, 3 Nov 2018 16:23:26 +0100
From:   Paweł Staszewski <pstaszewski@...are.pl>
To:     Jesper Dangaard Brouer <brouer@...hat.com>
Cc:     Aaron Lu <aaron.lu@...el.com>,
        Saeed Mahameed <saeedm@...lanox.com>,
        "eric.dumazet@...il.com" <eric.dumazet@...il.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        Tariq Toukan <tariqt@...lanox.com>,
        "ilias.apalodimas@...aro.org" <ilias.apalodimas@...aro.org>,
        "yoel@...knet.dk" <yoel@...knet.dk>,
        "mgorman@...hsingularity.net" <mgorman@...hsingularity.net>
Subject: Re: Kernel 4.19 network performance - forwarding/routing normal users
 traffic



W dniu 03.11.2018 o 13:58, Jesper Dangaard Brouer pisze:
> On Sat, 3 Nov 2018 01:16:08 +0100
> Paweł Staszewski <pstaszewski@...are.pl> wrote:
>
>> W dniu 02.11.2018 o 20:02, Paweł Staszewski pisze:
>>>
>>> W dniu 02.11.2018 o 15:20, Aaron Lu pisze:
>>>> On Fri, Nov 02, 2018 at 12:40:37PM +0100, Jesper Dangaard Brouer wrote:
>>>>> On Fri, 2 Nov 2018 13:23:56 +0800
>>>>> Aaron Lu <aaron.lu@...el.com> wrote:
>>>>>   
>>>>>> On Thu, Nov 01, 2018 at 08:23:19PM +0000, Saeed Mahameed wrote:
>>>>>>> On Thu, 2018-11-01 at 23:27 +0800, Aaron Lu wrote:
>>>>>>>> On Thu, Nov 01, 2018 at 10:22:13AM +0100, Jesper Dangaard Brouer
>>>>>>>> wrote:
>>>>>>>> ... ...
> [...]
>>>>> TL;DR: this is order-0 pages (code-walk trough proof below)
>>>>>
>>>>> To Aaron, the network stack *can* call __free_pages_ok() with order-0
>>>>> pages, via:
> [...]
>>>>   
>>>> I think here is a problem - order 0 pages are freed directly to buddy,
>>>> bypassing per-cpu-pages. This might be the reason lock contention
>>>> appeared on free path. Can someone apply below diff and see if lock
>>>> contention is gone?
>>>>
>>> Will test it tonight
>>>   
>> Patch applied
>> perf report:
>> https://ufile.io/sytfh
>>
>>
>> But i need to wait also with more traffic currently cpu's are sleeping
>>
> Well, that would be the expected result, that the CPUs get more time to
> sleep, if the lock contention is gone...
>
> What is the measured bandwidth now?
30 RX /30 TX Gbit/s

>
> Notice, you might still be limited by the PCIe bandwidth, but then your
> CPUs might actually decide to sleep, as they are getting data fast
> enough.
Yes - i will replace network controller to two separate nic's in two 
separate x16 pcie
But after monday.

But i dont think i hit pcie limit there - it looks like pcie x16 gen3 
have 16GB/s RX and 16GB/s TX so bidirectional


>
> [...]
>>>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>>>> index e2ef1c17942f..65c0ae13215a 100644
>>>> --- a/mm/page_alloc.c
>>>> +++ b/mm/page_alloc.c
>>>> @@ -4554,8 +4554,14 @@ void page_frag_free(void *addr)
>>>>    {
>>>>        struct page *page = virt_to_head_page(addr);
>>>>    -    if (unlikely(put_page_testzero(page)))
>>>> -        __free_pages_ok(page, compound_order(page));
>>>> +    if (unlikely(put_page_testzero(page))) {
>>>> +        unsigned int order = compound_order(page);
>>>> +
>>>> +        if (order == 0)
>>>> +            free_unref_page(page);
>>>> +        else
>>>> +            __free_pages_ok(page, order);
>>>> +    }
>
>