[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CANn89iJAKEUu_Fdh0OC-+BJ+iVY0D2y0nAakGLxWZ8TywDu=BA@mail.gmail.com>
Date: Wed, 28 Feb 2024 14:46:39 +0100
From: Eric Dumazet <edumazet@...gle.com>
To: Adam Li <adamli@...amperecomputing.com>
Cc: corbet@....net, davem@...emloft.net, kuba@...nel.org, pabeni@...hat.com,
willemb@...gle.com, yangtiezhu@...ngson.cn, atenart@...nel.org,
kuniyu@...zon.com, wuyun.abel@...edance.com, leitao@...ian.org,
alexander@...alicyn.com, dhowells@...hat.com, paulmck@...nel.org,
joel.granados@...il.com, urezki@...il.com, joel@...lfernandes.org,
linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
netdev@...r.kernel.org, patches@...erecomputing.com,
cl@...amperecomputing.com, shijie@...amperecomputing.com
Subject: Re: [PATCH] net: make SK_MEMORY_PCPU_RESERV tunable
On Wed, Feb 28, 2024 at 2:21 PM Adam Li <adamli@...amperecomputing.com> wrote:
>
> On 2/28/2024 4:38 AM, Eric Dumazet wrote:
>
> >>
> >> sk_prot->memory_allocated points to global atomic variable:
> >> atomic_long_t tcp_memory_allocated ____cacheline_aligned_in_smp;
> >>
> >> If increasing the per-cpu cache size from 1MB to e.g. 16MB,
> >> changes to sk->sk_prot->memory_allocated can be further reduced.
> >> Performance may be improved on system with many cores.
> >
> > This looks good, do you have any performance numbers to share ?
>
> I ran localhost memcached test on system with 320 CPU threads,
> perf shows 4% cycles spent in __sk_mem_raise_allocated() -->sk_memory_allocated().
> If increasing SK_MEMORY_PCPU_RESERV to 16MB, perf cycles spent in
> __sk_mem_raise_allocated() drops to 0.4%.
I suspect some kind of flow/cpu steering issues then.
Also maybe SO_RESERVE_MEM would be better for this workload.
Powered by blists - more mailing lists