netdev - Re: Kernel oops with mlx5 and dual XDP redirect programs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87efd5dill.fsf@toke.dk>
Date:   Thu, 04 Oct 2018 14:03:34 +0200
From:   Toke Høiland-Jørgensen <toke@...e.dk>
To:     Saeed Mahameed <saeedm@...lanox.com>,
        "netdev\@vger.kernel.org" <netdev@...r.kernel.org>
Cc:     Eran Ben Elisha <eranbe@...lanox.com>,
        Tariq Toukan <tariqt@...lanox.com>,
        "brouer\@redhat.com" <brouer@...hat.com>
Subject: Re: Kernel oops with mlx5 and dual XDP redirect programs

Saeed Mahameed <saeedm@...lanox.com> writes:

> On Wed, 2018-10-03 at 11:30 +0200, Toke Høiland-Jørgensen wrote:
>> Hi Saeed
>> 
>> I can reliably oops the kernel with the mlx5 driver, by installing
>> XDP_REDIRECT programs on two devices so they redirect to each other,
>> and then remove them while there is traffic on the interface.
>> 
>> Steps to reproduce:
>> 
>> # cd ~/build/linux/samples/bpf
>> # ./xdp_redirect_map $(</sys/class/net/ens1f1/ifindex)
>> $(</sys/class/net/ens1f0/ifindex)
>> # ./xdp_redirect_map $(</sys/class/net/ens1f0/ifindex)
>> $(</sys/class/net/ens1f1/ifindex)
>> 
>> Now, run some traffic (e.g., using pktgen) across the interfaces, and
>> while the traffic is running, interrupt one of the xdp_redirect_map
>> commands (thus unloading the eBPF program). This results in a kernel
>> oops with the backtrace below. I get no crash if there's only a
>> single
>> XDP program.
>
> Hi Toke,
>
> What looks like happening is that while the traffic is being redirected
> to the other device, the driver is trying to unload the program and
> restarting the rings from below call trace we can see:

Yeah, thought it was something like that, since it only happens on the
bidirectional redirect...

> I think that the mlx5 driver doesn't know how to tell the other device
> to stop transmitting to it while it is resetting.. Maybe tariq or
> Jesper know more about this ?
> I will look at this tomorrow after noon and will try to repro...

Great, thanks! :)

-Toke