netdev - Re: [PATCH net-next 0/4] page_pool: API for numa node change handling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 22 Oct 2019 09:33:04 -0700
From:   "Jonathan Lemon" <jonathan.lemon@...il.com>
To:     "Saeed Mahameed" <saeedm@...lanox.com>
Cc:     "David S. Miller" <davem@...emloft.net>,
        "Jesper Dangaard Brouer" <brouer@...hat.com>,
        netdev@...r.kernel.org, ilias.apalodimas@...aro.org
Subject: Re: [PATCH net-next 0/4] page_pool: API for numa node change handling

On 21 Oct 2019, at 21:44, Saeed Mahameed wrote:

> Hi Dave & Jesper,
>
> This series extends page pool API to allow page pool consumers to update
> page pool numa node on the fly. This is required since on some systems,
> rx rings irqs can migrate between numa nodes, due to irq balancer or user
> defined scripts, current page pool has no way to know of such migration
> and will keep allocating and holding on to pages from a wrong numa node,
> which is bad for the consumer performance.
>
> 1) Add API to update numa node id of the page pool
> Consumers will call this API to update the page pool numa node id.
>
> 2) Don't recycle non-reusable pages:
> Page pool will check upon page return whether a page is suitable for
> recycling or not.
>  2.1) when it belongs to a different num node.
>  2.2) when it was allocated under memory pressure.
>
> 3) mlx5 will use the new API to update page pool numa id on demand.
>
> The series is a joint work between me and Jonathan, we tested it and it
> proved itself worthy to avoid page allocator bottlenecks and improve
> packet rate and cpu utilization significantly for the described
> scenarios above.
>
> Performance testing:
> XDP drop/tx rate and TCP single/multi stream, on mlx5 driver
> while migrating rx ring irq from close to far numa:
>
> mlx5 internal page cache was locally disabled to get pure page pool
> results.
>
> CPU: Intel(R) Xeon(R) CPU E5-2603 v4 @ 1.70GHz
> NIC: Mellanox Technologies MT27700 Family [ConnectX-4] (100G)
>
> XDP Drop/TX single core:
> NUMA  | XDP  | Before    | After
> ---------------------------------------
> Close | Drop | 11   Mpps | 10.9 Mpps
> Far   | Drop | 4.4  Mpps | 5.8  Mpps
>
> Close | TX   | 6.5 Mpps  | 6.5 Mpps
> Far   | TX   | 4   Mpps  | 3.5  Mpps
>
> Improvement is about 30% drop packet rate, 15% tx packet rate for numa
> far test.
> No degradation for numa close tests.
>
> TCP single/multi cpu/stream:
> NUMA  | #cpu | Before  | After
> --------------------------------------
> Close | 1    | 18 Gbps | 18 Gbps
> Far   | 1    | 15 Gbps | 18 Gbps
> Close | 12   | 80 Gbps | 80 Gbps
> Far   | 12   | 68 Gbps | 80 Gbps
>
> In all test cases we see improvement for the far numa case, and no
> impact on the close numa case.

These look good, thanks Saeed!
-- 
Jonathan



>
> Thanks,
> Saeed.
>
> ---
>
> Jonathan Lemon (1):
>   page_pool: Restructure __page_pool_put_page()
>
> Saeed Mahameed (3):
>   page_pool: Add API to update numa node
>   page_pool: Don't recycle non-reusable pages
>   net/mlx5e: Rx, Update page pool numa node when changed
>
>  .../net/ethernet/mellanox/mlx5/core/en_rx.c   |  3 ++
>  include/net/page_pool.h                       |  7 +++
>  include/trace/events/page_pool.h              | 22 +++++++++
>  net/core/page_pool.c                          | 46 +++++++++++++------
>  4 files changed, 65 insertions(+), 13 deletions(-)
>
> -- 
> 2.21.0