Message-ID: <a122981d-ac9a-4c7e-a8a3-d50a3e613f0b@kernel.org>
Date: Thu, 19 Jun 2025 12:24:29 +0200
From: Jesper Dangaard Brouer <hawk@...nel.org>
To: Mina Almasry <almasrymina@...gle.com>
Cc: netdev@...r.kernel.org, linux-kernel@...r.kernel.org,
 linux-kselftest@...r.kernel.org, "David S. Miller" <davem@...emloft.net>,
 Eric Dumazet <edumazet@...gle.com>, Jakub Kicinski <kuba@...nel.org>,
 Paolo Abeni <pabeni@...hat.com>, Simon Horman <horms@...nel.org>,
 Shuah Khan <shuah@...nel.org>, Ilias Apalodimas
 <ilias.apalodimas@...aro.org>, Toke Høiland-Jørgensen
 <toke@...e.dk>, Ignat Korchagin <ignat@...udflare.com>
Subject: Re: [PATCH net-next v4] page_pool: import Jesper's page_pool
 benchmark



On 19/06/2025 06.19, Mina Almasry wrote:
> On Wed, Jun 18, 2025 at 5:46 AM Jesper Dangaard Brouer <hawk@...nel.org> wrote:
>>>> Something is off with benchmark numbers compared to the OOT version.
>>>>
>>>
>>> I assume you're comparing my results (my kernel config + my hardware +
>>> upstream benchmark) with your results (your kernel config + your
>>> hardware + OOT version). The problem may be in OOT vs upstream but it
>>> may be just different code/config/hardware.
>>
>> True I used OOT version.
>>
>> Just applied this patch, but I get compile error. Because Makefile tries
>> to get kernel headers (net/page_pool/helpers.h) from local Linux
>> installation instead of git tree.  This need to be adjusted for patch,
>> such that it builds with src-local/git tree provided headers.
>>
> 
> I believe the fix to that is to do:
> 
> make KDIR=$(pwd) -C ./tools/testing/selftests/net/bench
> 

Yes, this worked for me.

> I.e. the build files assume you're building the test to run it on the
> current machine, to cross compile it for a different machine under
> test, we need to pass explicit KDIR. I've kinda copy-pasted what other
> TEST_GEN_MODS_DIR= makefiles do. In theory we could do something else
> but I am guessing the way current TEST_GEN_MODS_DIR does it is the way
> to go. Does it work for you if you do that?

Yes.

> [...]
>>>
>>> Yeah, I actually just checked and I have CONFIG_DEBUG_NET on in my
>>> build, and a lot of other debug configs are turned on.
>>>
>>
>> The CONFIG_DEBUG_NET should be low overhead, so I don't expect this to
>> be the root-cause.  Other CONFIG options are more likely the issue.
>>
> 
> Thank you very much for the tips. Perf report showed the locking was
> taking forever on my kernel... I had locking debug configs enabled in
> my build... sorry... with those disabled, I get much more sane
> results:
> 
> [  185.557293] bench_page_pool: time_bench_page_pool01_fast_path():
> Cannot use page_pool fast-path
> [  185.607873] bench_page_pool: Type:no-softirq-page_pool01 Per elem: 11 cycles(tsc) 4.177 ns (step:0) - (measurement period
> time:0.041772642 sec time_interval:41772642) - (invoke count:10000000 tsc_interval:112778487)
> [  185.627090] bench_page_pool: time_bench_page_pool02_ptr_ring():
> Cannot use page_pool fast-path
> [  185.826991] bench_page_pool: Type:no-softirq-page_pool02 Per elem: 51 cycles(tsc) 19.117 ns (step:0) - (measurement period
> time:0.191178107 sec time_interval:191178107) - (invoke count:10000000 tsc_interval:516173586)
> [  185.846380] bench_page_pool: time_bench_page_pool03_slow(): Cannot
> use page_pool fast-path
> [  186.479432] bench_page_pool: Type:no-softirq-page_pool03 Per elem: 168 cycles(tsc) 62.469 ns (step:0) - (measurement period
> time:0.624690697 sec time_interval:624690697) - (invoke count:10000000 tsc_interval:1686656879)


My results with this patch:

$ sudo ./test_bench_page_pool.sh
rmmod: ERROR: Module bench_page_pool is not currently loaded
[268960.638885] bench_page_pool: Loaded
[268960.684603] bench_page_pool: Type:for_loop Per elem: 1 cycles(tsc) 
0.420 ns (step:0) - (measurement period time:0.042037752 sec 
time_interval:42037752) - (invoke count:100000000 tsc_interval:151336355)
[268961.203806] bench_page_pool: Type:atomic_inc Per elem: 18 
cycles(tsc) 5.010 ns (step:0) - (measurement period time:0.501077936 sec 
time_interval:501077936) - (invoke count:100000000 tsc_interval:1803899823)
[268961.332771] bench_page_pool: Type:lock Per elem: 39 cycles(tsc) 
11.041 ns (step:0) - (measurement period time:0.110410468 sec 
time_interval:110410468) - (invoke count:10000000 tsc_interval:397481145)
[268961.350718] bench_page_pool: time_bench_page_pool01_fast_path(): 
Cannot use page_pool fast-path
[268961.425335] bench_page_pool: Type:no-softirq-page_pool01 Per elem: 
23 cycles(tsc) 6.571 ns (step:0) - (measurement period time:0.065719390 
sec time_interval:65719390) - (invoke count:10000000 tsc_interval:236591475)
[268961.444666] bench_page_pool: time_bench_page_pool02_ptr_ring(): 
Cannot use page_pool fast-path
[268961.622103] bench_page_pool: Type:no-softirq-page_pool02 Per elem: 
60 cycles(tsc) 16.862 ns (step:0) - (measurement period time:0.168626201 
sec time_interval:168626201) - (invoke count:10000000 
tsc_interval:607060218)
[268961.641608] bench_page_pool: time_bench_page_pool03_slow(): Cannot 
use page_pool fast-path
[268962.387479] bench_page_pool: Type:no-softirq-page_pool03 Per elem: 
265 cycles(tsc) 73.739 ns (step:0) - (measurement period 
time:0.737399722 sec time_interval:737399722) - (invoke count:10000000 
tsc_interval:2654665224)

Fast path results:
no-softirq-page_pool01 Per elem: 23 cycles(tsc) 6.571 ns

ptr_ring results:
no-softirq-page_pool02 Per elem: 60 cycles(tsc) 16.862 ns

slow path results:
no-softirq-page_pool03 Per elem: 265 cycles(tsc) 73.739 ns


> Does this alleviate your concern? Or do you still see an issue here?
> There is still a delta between our results, on different
> hardware/configs but results are in a sane range now.

Now the results a sane and in range :-)

--Jesper

Powered by blists - more mailing lists