netdev - Re: [RFC] change the default Kconfig value of mlx5

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAA85sZsH4JNczUzK0EfGC959y=+SK1b=T6_5zb_0e1b99xGtvA@mail.gmail.com>
Date:   Sat, 22 Apr 2017 11:28:06 +0200
From:   Ian Kumlien <ian.kumlien@...il.com>
To:     Saeed Mahameed <saeedm@....mellanox.co.il>
Cc:     Leon Romanovsky <leonro@...lanox.com>,
        Matan Barak <matanb@...lanox.com>,
        David Miller <davem@...emloft.net>,
        Saeed Mahameed <saeedm@...lanox.com>,
        Linux Kernel Network Developers <netdev@...r.kernel.org>
Subject: Re: [RFC] change the default Kconfig value of mlx5_en

On Sat, Apr 22, 2017 at 3:07 AM, Saeed Mahameed
<saeedm@....mellanox.co.il> wrote:
> On Sat, Apr 22, 2017 at 3:47 AM, Ian Kumlien <ian.kumlien@...il.com> wrote:
>> On Sat, Apr 22, 2017 at 2:34 AM, Saeed Mahameed
>> <saeedm@....mellanox.co.il> wrote:
>>> On Sat, Apr 22, 2017 at 2:10 AM, Ian Kumlien <ian.kumlien@...il.com> wrote:
>>>> Sorry,
>>>>
>>>> Back again, fighting cold, hot whiskey has been consumed...
>>>>
>>>> Something like this would perhaps be a better solution:
>>>>
>>>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c
>>>> b/drivers/net/ethernet/mellanox/mlx5/core/main.c
>>>> index 60154a175bd3..fe192e247601 100644
>>>> --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c
>>>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c
>>>> @@ -1139,6 +1139,10 @@ static int mlx5_load_one(struct mlx5_core_dev
>>>> *dev, struct mlx5_priv *priv,
>>>>
>>>>  #ifdef CONFIG_MLX5_CORE_EN
>>>>         mlx5_eswitch_attach(dev->priv.eswitch);
>>>> +#else
>>>> +       if (MLX5_CAP_GEN(dev, port_type) == MLX5_CAP_PORT_TYPE_ETH) {
>>>> +               dev_info(&pdev->dev, "Ethernet device discovered but
>>>> support not enabled in kernel.");
>>>> +       }
>>>>  #endif
>>>>
>>>
>>> Currently both MLX5_CORE=n and MLX5_CORE_EN=n as a default, the issue
>>> you are seeing can occur only if you explicitly  set MLX5_CORE=y and
>>> MLX5_CORE=n, Why would someone do this if he knows he wants Ethernet
>>> support as well ? IMHO this print is redundant .
>>
>> Well, I'm running a prebuilt kernel - which was configured this way,
>> and since there
>> is no mlx5_en module and it does state that the link is "Ethernet", it
>> just looks like the
>> driver is broken or in some kind of really weird state.
>>
>>> Anyway, Are you looking for RDMA support over ethernet (RoCE) ? and
>>> you are not interested to have ethernet netdev support ?
>>
>> ? RDMA is something we'll look at in the future, right now, having the
>> nics actually
>> work as nics is a priority ;)
>>
>
> I see, i just wanted to understand your situation :)
>
>>> if yes, I think this is something that can be achieved, but the
>>> question is do we really need this ?
>>
>> It's really weird to see the driver load, to see everything register
>> and have no feedback.
>>
>
> So, in your case you have mlx5 core support without MLX5_CORE_EN which
> provides the eswitch and netdev functionality in ethernet.

Yes

> But you will still have mlx5_ib register an RDMA interface and
> theoretically it should work, the only thing you won't see is a
> netdevice.
>
> The weird thing is that you don't see a link up on the RDMA interface,
> Leon/Matan can you please look into this ? do we really need a netdev
> to have a functioning RDMA logical link in ethernet ?

The switch we have does support RDMA but the manual is sparse (as in
nothing really there) wrt enabling/configuring the RDMA bit so something
might be missing.

I'll try to remember to do the same test when we setup the mellanox switches =)

>> Including no network devices, but if you run the Infiniband commands,
>> they tell you that
>> you are connected to Ethernet but that the device is down and disabled.
>>
>> To me, down and disabled is not the same as in "Ethernet support is
>> not included" =)
>>
>> Basically, i would hate for someone else to end up in the same
>> situation since you only
>> get guides on how to enable infiniband/RDMA but what you really want
>> to do at that point
>> is to disable it and see if that gives you your network devices back =)
>>
>
> Yes this is misleading, Maybe your kernel log warning is not so bad
> after all, but let me dig more into this.
> I will get back to you next week.

Thanks, I bet that there is better ways to do it, this one was just
one of the first ones i found =)

>> I have had similar issues with some connectx3 devices while playing at
>> home but i suspect
>> that it's just a limitation of OFED packages available for the dist I'm running.