netdev - Re: Resource management for ndo_xdp_xmit (Was: [PATCH net] virtio_net: Account for tx bytes and packets on sending xdp

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <9e5e6882566ac67276209b35ec112a824b256bff.camel@mellanox.com>
Date:   Thu, 7 Feb 2019 19:08:14 +0000
From:   Saeed Mahameed <saeedm@...lanox.com>
To:     "brouer@...hat.com" <brouer@...hat.com>
CC:     "thoiland@...hat.com" <thoiland@...hat.com>,
        "hawk@...nel.org" <hawk@...nel.org>,
        "virtualization@...ts.linux-foundation.org" 
        <virtualization@...ts.linux-foundation.org>,
        "borkmann@...earbox.net" <borkmann@...earbox.net>,
        Tariq Toukan <tariqt@...lanox.com>,
        "john.fastabend@...il.com" <john.fastabend@...il.com>,
        "toke@...e.dk" <toke@...e.dk>, "mst@...hat.com" <mst@...hat.com>,
        "jakub.kicinski@...ronome.com" <jakub.kicinski@...ronome.com>,
        "dsahern@...il.com" <dsahern@...il.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "jasowang@...hat.com" <jasowang@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "makita.toshiaki@....ntt.co.jp" <makita.toshiaki@....ntt.co.jp>
Subject: Re: Resource management for ndo_xdp_xmit (Was: [PATCH net]
 virtio_net: Account for tx bytes and packets on sending xdp_frames)

On Thu, 2019-02-07 at 08:48 +0100, Jesper Dangaard Brouer wrote:
> On Wed, 6 Feb 2019 00:06:33 +0000 Saeed Mahameed <saeedm@...lanox.com
> > wrote:
> 
> > On Mon, 2019-02-04 at 19:13 -0800, David Ahern wrote:
> [...]
> > > mlx5 needs some work. As I recall it still has the bug/panic
> > > removing xdp programs - at least I don't recall seeing a patch
> > > for
> > > it.  
> > 
> > Only when xdp_redirect to mlx5, and removing the program while
> > redirect is happening, this is actually due to a lack of
> > synchronization means between different drivers, we have some ideas
> > to overcome this using a standard XDP API, or just use a hack in
> > mlx5
> > driver which i don't like:
> > 
> > https://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git/commit/?h=topic/xdp-redirect-fix&id=a3652d03cc35fd3ad62744986c8ccaca74c9f20c
> > 
> > I will be working on this towards the end of this week.
> 
> Toke and I have been discussing how to solve this.
> 
> The main idea for fixing this is to tie resource allocation to
> interface
> insertion into interface maps (kernel/bpf/devmap.c). As the =devmap=
> already have the needed synchronisation mechanisms and steps for
> safely
> adding and removing =net_devices= (e.g. stopping RX side, flushing
> remaining frames, waiting RCU period before freeing objects, etc.)
> 
> As described here:
>  
> https://github.com/xdp-project/xdp-project/blob/master/xdp-project.org#better-ndo_xdp_xmit-resource-management
> 
> --Jesper

Yes you already suggested this approach @LPC:

So 
1) on dev_map_update_elem() we will call
dev->dev->ndo_bpf() to notify the device on the intention to start/stop
redirect, and wait for it to create/destroy the HW resources
before/after actually updating the map

But:
2) this won't totally solve our problem, since sometimes the driver can
decide to recreate (change of configuration) hw resources on the fly
while redirect/devmap is already happening, so we need some kind of a
dev_map_notification or a flag with rcu synch, for when the driver want
to make the xdp redirect resources unavailable.

Thanks,
Saeed.