netdev - Re: Optimizing kernel compilation / alignments for network performance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <2c7e6c5f-1006-3f51-362d-249dd12b5547@gmail.com>
Date:   Sun, 8 May 2022 11:53:23 +0200
From:   Rafał Miłecki <zajec5@...il.com>
To:     Andrew Lunn <andrew@...n.ch>
Cc:     Arnd Bergmann <arnd@...db.de>,
        Alexander Lobakin <alexandr.lobakin@...el.com>,
        Network Development <netdev@...r.kernel.org>,
        linux-arm-kernel <linux-arm-kernel@...ts.infradead.org>,
        Russell King <linux@...linux.org.uk>,
        Felix Fietkau <nbd@....name>,
        "openwrt-devel@...ts.openwrt.org" <openwrt-devel@...ts.openwrt.org>,
        Florian Fainelli <f.fainelli@...il.com>
Subject: Re: Optimizing kernel compilation / alignments for network
 performance

On 6.05.2022 09:44, Rafał Miłecki wrote:
> On 5.05.2022 18:04, Andrew Lunn wrote:
>>> you'll see that most used functions are:
>>> v7_dma_inv_range
>>> __irqentry_text_end
>>> l2c210_inv_range
>>> v7_dma_clean_range
>>> bcma_host_soc_read32
>>> __netif_receive_skb_core
>>> arch_cpu_idle
>>> l2c210_clean_range
>>> fib_table_lookup
>>
>> There is a lot of cache management functions here. Might sound odd,
>> but have you tried disabling SMP? These cache functions need to
>> operate across all CPUs, and the communication between CPUs can slow
>> them down. If there is only one CPU, these cache functions get simpler
>> and faster.
>>
>> It just depends on your workload. If you have 1 CPU loaded to 100% and
>> the other 3 idle, you might see an improvement. If you actually need
>> more than one CPU, it will probably be worse.
> 
> It seems to lower my NAT speed from ~362 Mb/s to 320 Mb/s but it feels
> more stable now (lower variations). Let me spend some time on more
> testing.

For a context I test various kernel commits / configs using:
iperf -t 120 -i 10 -c 192.168.13.1


I did more testing with # CONFIG_SMP is not set

Good thing:
During a single iperf session I get noticably more stable speed.
With SMP: x ± 2,86%
Without SMP: x ± 0,96%

Bad thing:
Across kernel commits / config changes speed still varies.


So disabling CONFIG_SMP won't help me looking for kernel regressions.