| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <87ikmj5bh5.fsf@toke.dk>
Date: Fri, 02 May 2025 16:33:10 +0200
From: Toke Høiland-Jørgensen <toke@...hat.com>
To: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc: netdev@...r.kernel.org, linux-rt-devel@...ts.linux.dev, "David S.
Miller" <davem@...emloft.net>, Eric Dumazet <edumazet@...gle.com>, Jakub
Kicinski <kuba@...nel.org>, Paolo Abeni <pabeni@...hat.com>, Simon Horman
<horms@...nel.org>, Thomas Gleixner <tglx@...utronix.de>, Andrew Lunn
<andrew+netdev@...n.ch>, Alexei Starovoitov <ast@...nel.org>, Daniel
Borkmann <daniel@...earbox.net>, Jesper Dangaard Brouer <hawk@...nel.org>,
John Fastabend <john.fastabend@...il.com>
Subject: Re: [PATCH net-next v3 05/18] xdp: Use nested-BH locking for
system_page_pool
Sebastian Andrzej Siewior <bigeasy@...utronix.de> writes:
> On 2025-05-01 12:13:24 [+0200], Toke Høiland-Jørgensen wrote:
>> > --- a/net/core/dev.c
>> > +++ b/net/core/dev.c
>> > @@ -462,7 +462,9 @@ EXPORT_PER_CPU_SYMBOL(softnet_data);
>> > * PP consumers must pay attention to run APIs in the appropriate context
>> > * (e.g. NAPI context).
>> > */
>> > -DEFINE_PER_CPU(struct page_pool *, system_page_pool);
>> > +DEFINE_PER_CPU(struct page_pool_bh, system_page_pool) = {
>> > + .bh_lock = INIT_LOCAL_LOCK(bh_lock),
>> > +};
>>
>> I'm a little fuzzy on how DEFINE_PER_CPU() works, but does this
>> initialisation automatically do the right thing with the multiple
>> per-CPU instances?
>
> It sets the "first" per-CPU data which is then copied to all
> "possible-CPUs" during early boot when the per-CPU data is made
> available. You can initialize almost everything like that. Pointer based
> structures (such as LIST_HEAD_INIT()) is something that obviously won't
> work.
Right, I see. Cool, thanks for explaining :)
>> > #ifdef CONFIG_LOCKDEP
>> > /*
>> > --- a/net/core/xdp.c
>> > +++ b/net/core/xdp.c
>> > @@ -737,10 +737,10 @@ static noinline bool xdp_copy_frags_from_zc(struct sk_buff *skb,
>> > */
>> > struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
>> > {
>> > - struct page_pool *pp = this_cpu_read(system_page_pool);
>> > const struct xdp_rxq_info *rxq = xdp->rxq;
>> > u32 len = xdp->data_end - xdp->data_meta;
>> > u32 truesize = xdp->frame_sz;
>> > + struct page_pool *pp;
>> > struct sk_buff *skb;
>> > int metalen;
>> > void *data;
>> > @@ -748,13 +748,18 @@ struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
>> > if (!IS_ENABLED(CONFIG_PAGE_POOL))
>> > return NULL;
>> >
>> > + local_lock_nested_bh(&system_page_pool.bh_lock);
>> > + pp = this_cpu_read(system_page_pool.pool);
>> > data = page_pool_dev_alloc_va(pp, &truesize);
>> > - if (unlikely(!data))
>> > + if (unlikely(!data)) {
>> > + local_unlock_nested_bh(&system_page_pool.bh_lock);
>> > return NULL;
>> > + }
>> >
>> > skb = napi_build_skb(data, truesize);
>> > if (unlikely(!skb)) {
>> > page_pool_free_va(pp, data, true);
>> > + local_unlock_nested_bh(&system_page_pool.bh_lock);
>> > return NULL;
>> > }
>> >
>> > @@ -773,9 +778,11 @@ struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
>> >
>> > if (unlikely(xdp_buff_has_frags(xdp)) &&
>> > unlikely(!xdp_copy_frags_from_zc(skb, xdp, pp))) {
>> > + local_unlock_nested_bh(&system_page_pool.bh_lock);
>> > napi_consume_skb(skb, true);
>> > return NULL;
>> > }
>> > + local_unlock_nested_bh(&system_page_pool.bh_lock);
>>
>> Hmm, instead of having four separate unlock calls in this function, how
>> about initialising skb = NULL, and having the unlock call just above
>> 'return skb' with an out: label?
>>
>> Then the three topmost 'return NULL' can just straight-forwardly be
>> replaced with 'goto out', while the last one becomes 'skb = NULL; goto
>> out;'. I think that would be more readable than this repetition.
>
> Something like the following maybe? We would keep the lock during
> napi_consume_skb() which should work.
>
> diff --git a/net/core/xdp.c b/net/core/xdp.c
> index b2a5c934fe7b7..1ff0bc328305d 100644
> --- a/net/core/xdp.c
> +++ b/net/core/xdp.c
> @@ -740,8 +740,8 @@ struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
> const struct xdp_rxq_info *rxq = xdp->rxq;
> u32 len = xdp->data_end - xdp->data_meta;
> u32 truesize = xdp->frame_sz;
> + struct sk_buff *skb = NULL;
> struct page_pool *pp;
> - struct sk_buff *skb;
> int metalen;
> void *data;
>
> @@ -751,16 +751,13 @@ struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
> local_lock_nested_bh(&system_page_pool.bh_lock);
> pp = this_cpu_read(system_page_pool.pool);
> data = page_pool_dev_alloc_va(pp, &truesize);
> - if (unlikely(!data)) {
> - local_unlock_nested_bh(&system_page_pool.bh_lock);
> - return NULL;
> - }
> + if (unlikely(!data))
> + goto out;
>
> skb = napi_build_skb(data, truesize);
> if (unlikely(!skb)) {
> page_pool_free_va(pp, data, true);
> - local_unlock_nested_bh(&system_page_pool.bh_lock);
> - return NULL;
> + goto out;
> }
>
> skb_mark_for_recycle(skb);
> @@ -778,15 +775,16 @@ struct sk_buff *xdp_build_skb_from_zc(struct xdp_buff *xdp)
>
> if (unlikely(xdp_buff_has_frags(xdp)) &&
> unlikely(!xdp_copy_frags_from_zc(skb, xdp, pp))) {
> - local_unlock_nested_bh(&system_page_pool.bh_lock);
> napi_consume_skb(skb, true);
> - return NULL;
> + skb = NULL;
> }
> +
> +out:
> local_unlock_nested_bh(&system_page_pool.bh_lock);
> -
> - xsk_buff_free(xdp);
> -
> - skb->protocol = eth_type_trans(skb, rxq->dev);
> + if (skb) {
> + xsk_buff_free(xdp);
> + skb->protocol = eth_type_trans(skb, rxq->dev);
> + }
I had in mind moving the out: label (and the unlock) below the
skb->protocol assignment, which would save the if(skb) check; any reason
we can't call xsk_buff_free() while holding the lock?
-Toke
Powered by blists - more mailing lists