Date:   Thu, 6 Jun 2019 10:23:40 +0000
From:   "Jubran, Samih" <sameehj@...zon.com>
To:     Jakub Kicinski <jakub.kicinski@...ronome.com>,
        David Woodhouse <dwmw2@...radead.org>
CC:     "Bshara, Nafea" <nafea@...zon.com>, Andrew Lunn <andrew@...n.ch>,
        "Kiyanovski, Arthur" <akiyano@...zon.com>,
        "Bshara, Saeed" <saeedb@...zon.com>,
        "Tzalik, Guy" <gtzalik@...zon.com>,
        "Matushevsky, Alexander" <matua@...zon.com>,
        "Liguori, Anthony" <aliguori@...zon.com>,
        "Saidi, Ali" <alisaidi@...zon.com>,
        "Machulsky, Zorik" <zorik@...zon.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "Wilson, Matt" <msw@...zon.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "Belgazal, Netanel" <netanel@...zon.com>,
        "Herrenschmidt, Benjamin" <benh@...zon.com>
Subject: RE: [PATCH V2 net 00/11] Extending the ena driver to support new
 features and enhance performance

As of today there are no flags exposed by ENA NIC device, however, we are planning to use them in the near future.
We want to provide customers with extra methods to identify (and differentiate) multiple network interfaces that can be attached to a single VM. 
Currently, customers can identify a specific network interface (ENI) only by MAC address, and later look up this MAC among other multiple ENIs that they have.
In some cases it might not be convenient. Using these flags will let us inform the customer about ENI`s specific properties. It's not finalized,
but tentatively it can look like this:
primary-eni: on /* Differentiate between primary and secondary ENIs */
has-associated-efa: on /* Will specify ENA device has another associated EFA device */

> -----Original Message-----
> From: Jakub Kicinski <jakub.kicinski@...ronome.com>
> Sent: Tuesday, June 4, 2019 8:24 PM
> To: David Woodhouse <dwmw2@...radead.org>
> Cc: Bshara, Nafea <nafea@...zon.com>; Andrew Lunn <andrew@...n.ch>;
> Jubran, Samih <sameehj@...zon.com>; Kiyanovski, Arthur
> <akiyano@...zon.com>; Bshara, Saeed <saeedb@...zon.com>; Tzalik,
> Guy <gtzalik@...zon.com>; Matushevsky, Alexander
> <matua@...zon.com>; Liguori, Anthony <aliguori@...zon.com>; Saidi, Ali
> <alisaidi@...zon.com>; Machulsky, Zorik <zorik@...zon.com>;
> netdev@...r.kernel.org; Wilson, Matt <msw@...zon.com>;
> davem@...emloft.net; Belgazal, Netanel <netanel@...zon.com>;
> Herrenschmidt, Benjamin <benh@...zon.com>
> Subject: Re: [PATCH V2 net 00/11] Extending the ena driver to support new
> features and enhance performance
> 
> On Tue, 04 Jun 2019 07:57:48 +0100, David Woodhouse wrote:
> > On Tue, 2019-06-04 at 02:15 +0000, Bshara, Nafea wrote:
> > > On Jun 3, 2019, at 6:52 PM, Andrew Lunn <andrew@...n.ch> wrote:
> > > > > Any "SmartNIC" vendor has temptation of uAPI-level hand off to
> > > > > the firmware (including my employer), we all run pretty beefy
> > > > > processors inside "the NIC" after all.  The device centric
> > > > > ethtool configuration can be implemented by just forwarding the
> > > > > uAPI structures as they are to the FW.  I'm sure Andrew and
> > > > > others who would like to see Linux takes more control over PHYs etc.
> would not like this scenario, either.
> > > >
> > > > No, i would not. There are a few good examples of both firmware
> > > > and open drivers being used to control the same PHY, on different
> > > > boards. The PHY driver was developed by the community, and has
> > > > more features than the firmware driver. And it keeps gaining
> > > > features. The firmware i stuck, no updates. The community driver
> > > > can be debugged, the firmware is a black box, no chance of the
> > > > community fixing any bugs in it.
> > > >
> > > > And PHYs are commodity devices. I doubt there is any value add in
> > > > the firmware for a PHY, any real IPR which makes the product
> > > > better, magic sauce related to the PHY. So just save the cost of
> > > > writing and maintaining firmware, export the MDIO bus, and let Linux
> control it.
> > > > Concentrate the engineers on the interesting parts of the NIC, the
> > > > Smart parts, where there can be real IPR.
> > > >
> > > > And i would say this is true for any NIC. Let Linux control the PHY.
> > >
> > > It may be true for old GbE PHYs where it’s a discrete chip from the
> > > likes of Marvell or broadcom
> > >
> > > But at 25/50/100G, the PHy is actually part of the nic. It’s a very
> > > complex SERDES.  Cloud providers like us spend enormous amount of
> > > time testing the PHY across process and voltage variations, all
> > > cable types, length and manufacturing variations, and against all
> > > switches we use.  Community drivers won’t be able to validate and
> > > tune all this.
> > >
> > > Plus we would need exact same setting for Linux, including all
> > > distributions even 10year old like RHEL6, for all Windows, ESX,
> > > DPDK, FreeBSD,  and support millions of different customers with
> > > different sets of Machine images.
> > >
> > > In this case, there is no practical choice by have the firmware to
> > > manage the PHY
> >
> > I don't quite know why we're talking about PHYs in this context.
> > ENA is basically a virtio NIC. It has no PHY.
> 
> I brought it up as an example, to illustrate that we'd rather see less trust and
> control being blindly handed over to the firmware.
> 
> Would you mind answering what are the examples of very important flags
> you need to expose to users with 10yo kernels?  Or any examples for that
> matter..

Powered by blists - more mailing lists