[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <13b616bb-52f1-426e-8529-0bbb7b279d89@gmail.com>
Date: Mon, 18 Mar 2024 14:14:32 +0000
From: Pavel Begunkov <asml.silence@...il.com>
To: Eric Dumazet <edumazet@...gle.com>
Cc: netdev@...r.kernel.org, davem@...emloft.net, dsahern@...nel.org,
pabeni@...hat.com, kuba@...nel.org
Subject: Re: [PATCH net-next v2] net: cache for same cpu
skb_attempt_defer_free
On 3/18/24 11:41, Pavel Begunkov wrote:
> On 3/18/24 10:11, Eric Dumazet wrote:
>> On Mon, Mar 18, 2024 at 1:46 AM Pavel Begunkov <asml.silence@...il.com> wrote:
>>>
>>> Optimise skb_attempt_defer_free() when run by the same CPU the skb was
>>> allocated on. Instead of __kfree_skb() -> kmem_cache_free() we can
>>> disable softirqs and put the buffer into cpu local caches.
>>>
>>> CPU bound TCP ping pong style benchmarking (i.e. netbench) showed a 1%
>>> throughput increase (392.2 -> 396.4 Krps). Cross checking with profiles,
>>> the total CPU share of skb_attempt_defer_free() dropped by 0.6%. Note,
>>> I'd expect the win doubled with rx only benchmarks, as the optimisation
>>> is for the receive path, but the test spends >55% of CPU doing writes.
>>>
>>> Signed-off-by: Pavel Begunkov <asml.silence@...il.com>
>>> ---
>>>
>>> v2: pass @napi_safe=true by using __napi_kfree_skb()
>>>
>>> net/core/skbuff.c | 15 ++++++++++++++-
>>> 1 file changed, 14 insertions(+), 1 deletion(-)
>>>
>>> diff --git a/net/core/skbuff.c b/net/core/skbuff.c
>>> index b99127712e67..35d37ae70a3d 100644
>>> --- a/net/core/skbuff.c
>>> +++ b/net/core/skbuff.c
>>> @@ -6995,6 +6995,19 @@ void __skb_ext_put(struct skb_ext *ext)
>>> EXPORT_SYMBOL(__skb_ext_put);
>>> #endif /* CONFIG_SKB_EXTENSIONS */
>>>
>>> +static void kfree_skb_napi_cache(struct sk_buff *skb)
>>> +{
>>> + /* if SKB is a clone, don't handle this case */
>>> + if (skb->fclone != SKB_FCLONE_UNAVAILABLE) {
>>> + __kfree_skb(skb);
>>> + return;
>>> + }
>>> +
>>> + local_bh_disable();
>>> + __napi_kfree_skb(skb, SKB_DROP_REASON_NOT_SPECIFIED);
>>> + local_bh_enable();
>>> +}
>>> +
>>> /**
>>> * skb_attempt_defer_free - queue skb for remote freeing
>>> * @skb: buffer
>>> @@ -7013,7 +7026,7 @@ void skb_attempt_defer_free(struct sk_buff *skb)
>>> if (WARN_ON_ONCE(cpu >= nr_cpu_ids) ||
>>> !cpu_online(cpu) ||
>>> cpu == raw_smp_processor_id()) {
>>> -nodefer: __kfree_skb(skb);
>>> +nodefer: kfree_skb_napi_cache(skb);
>>> return;
>>> }
>>>
>>> --
>>> 2.44.0
>>>
>>
>> 1) net-next is currently closed.
>
> Ok
>
>> 2) No NUMA awareness. SLUB does not guarantee the sk_buff was on the
>> correct node.
>
> Let me see if I read you right. You're saying that SLUB can
> allocate an skb from a different node, so skb->alloc_cpu
> might be not indicative of the node, and so we might locally
> cache an skb of a foreign numa node?
>
> If that's the case I don't see how it's different from the
> cpu != raw_smp_processor_id() path, which will transfer the
> skb to another cpu and still put it in the local cache in
> softirq.
>
>
>> 3) Given that many skbs (like TCP ACK) are freed using __kfree_skb(), I wonder
>> why trying to cache the sk_buff in this particular path is needed.
>>
>> Why not change __kfree_skb() instead ?
>
> IIRC kfree_skb() can be called from any context including irqoff,
On the other hand there is napi_pp_put_page() inside which
assumes hardirq off, so maybe not, I'd appreciate if someone
can clarify it. I was sure that drivers allocating in hardirq
via e.g. __netdev_alloc_skb() might want to use kfree_skb(),
but maybe they're consistently sticking to dev_kfree_sk*().
> it's convenient to have a function that just does the job without
> too much of extra care. Theoretically it can have a separate path
> inside based on irqs_disabled(), but that would be ugly.
--
Pavel Begunkov
Powered by blists - more mailing lists