netdev - Re: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce mlx4_xdp_buff wrapper for xdp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <874jun3m58.fsf@toke.dk>
Date:   Fri, 25 Nov 2022 01:36:51 +0100
From:   Toke Høiland-Jørgensen <toke@...hat.com>
To:     Maciej Fijalkowski <maciej.fijalkowski@...el.com>
Cc:     Jakub Kicinski <kuba@...nel.org>, sdf@...gle.com,
        bpf@...r.kernel.org, ast@...nel.org, daniel@...earbox.net,
        andrii@...nel.org, martin.lau@...ux.dev, song@...nel.org,
        yhs@...com, john.fastabend@...il.com, kpsingh@...nel.org,
        haoluo@...gle.com, jolsa@...nel.org,
        Tariq Toukan <tariqt@...dia.com>,
        David Ahern <dsahern@...il.com>,
        Willem de Bruijn <willemb@...gle.com>,
        Jesper Dangaard Brouer <brouer@...hat.com>,
        Anatoly Burakov <anatoly.burakov@...el.com>,
        Alexander Lobakin <alexandr.lobakin@...el.com>,
        Magnus Karlsson <magnus.karlsson@...il.com>,
        Maryam Tahhan <mtahhan@...hat.com>, xdp-hints@...-project.net,
        netdev@...r.kernel.org
Subject: Re: [xdp-hints] Re: [PATCH bpf-next v2 6/8] mlx4: Introduce
 mlx4_xdp_buff wrapper for xdp_buff

Maciej Fijalkowski <maciej.fijalkowski@...el.com> writes:

> On Thu, Nov 24, 2022 at 04:17:01PM +0100, Maciej Fijalkowski wrote:
>> On Thu, Nov 24, 2022 at 03:39:20PM +0100, Toke Høiland-Jørgensen wrote:
>> > Jakub Kicinski <kuba@...nel.org> writes:
>> > 
>> > > On Wed, 23 Nov 2022 22:55:21 +0100 Toke Høiland-Jørgensen wrote:
>> > >> > Good idea, prototyped below, lmk if it that's not what you had in mind.
>> > >> >
>> > >> > struct xdp_buff_xsk {
>> > >> > 	struct xdp_buff            xdp;                  /*     0    56 */
>> > >> > 	u8                         cb[16];               /*    56    16 */
>> > >> > 	/* --- cacheline 1 boundary (64 bytes) was 8 bytes ago --- */  
>> > >> 
>> > >> As pahole helpfully says here, xdp_buff is actually only 8 bytes from
>> > >> being a full cache line. I thought about adding a 'cb' field like this
>> > >> to xdp_buff itself, but figured that since there's only room for a
>> > >> single pointer, why not just add that and let the driver point it to
>> > >> where it wants to store the extra context data?
>> > >
>> > > What if the driver wants to store multiple pointers or an integer or
>> > > whatever else? The single pointer seems quite arbitrary and not
>> > > strictly necessary.
>> > 
>> > Well, then you allocate a separate struct and point to that? Like I did
>> > in mlx5:
>> > 
>> > 
>> > +	struct mlx5_xdp_ctx mlctx = { .cqe = cqe, .rq = rq };
>> > +	struct xdp_buff xdp = { .drv_priv = &mlctx };
>> > 
>> > but yeah, this does give an extra pointer deref on access. I'm not
>> > really opposed to the cb field either, I just think it's a bit odd to
>> > put it in struct xdp_buff_xsk; that basically requires the driver to
>> > keep the layouts in sync.
>> > 
>> > Instead, why not but a cb field into xdp_buff itself so it can be used
>> > for both the XSK and the non-XSK paths? Then the driver can just
>> > typecast the xdp_buff into its own struct that has whatever data it
>> > wants in place of the cb field?
>> 
>> Why can't you simply have a pointer to xdp_buff in driver specific
>> xdp_buff container which would point to xdp_buff that is stack based (or
>> whatever else memory that will back it up - I am about to push a change
>> that makes ice driver embed xdp_buff within a struct that represents Rx
>> ring) for XDP path and for ZC the pointer to xdp_buff that you get from
>> xsk_buff_pool ? This would satisfy both sides I believe and would let us
>> keep the same container struct.
>> 
>> struct mlx4_xdp_buff {
>> 	struct xdp_buff *xdp;
>> 	struct mlx4_cqe *cqe;
>> 	struct mlx4_en_dev *mdev;
>> 	struct mlx4_en_rx_ring *ring;
>> 	struct net_device *dev;
>> };
>
> Nah this won't work from kfunc POV, probably no way to retrieve the
> mlx4_xdp_buff based on xdp_buff ptr that needs to be used as an arg.
>
> Sorry I'll think more about it, in the meantime let's hear more voices
> whether we should keep Stan's original approach + modify xdp_buff_xsk or
> go with Toke's proposal.

OK, so I played around with the mlx5 code a bit more, and I think the
"wrapping struct + cb area" can be made to work without too many ugly
casts; I'll send an updated version of the mlx5 patches with this
incorporated tomorrow, after I've run some tests...

-Toke