Message-ID: 
 <CH0PR18MB4339666BBD106C2EEF590E57CDDF2@CH0PR18MB4339.namprd18.prod.outlook.com>
Date: Fri, 5 Jul 2024 11:12:45 +0000
From: Geethasowjanya Akula <gakula@...vell.com>
To: Michal Swiatkowski <michal.swiatkowski@...ux.intel.com>
CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "kuba@...nel.org" <kuba@...nel.org>,
        "davem@...emloft.net"
	<davem@...emloft.net>,
        "pabeni@...hat.com" <pabeni@...hat.com>,
        "edumazet@...gle.com" <edumazet@...gle.com>,
        Sunil Kovvuri Goutham
	<sgoutham@...vell.com>,
        Subbaraya Sundeep Bhatta <sbhatta@...vell.com>,
        Hariprasad Kelam <hkelam@...vell.com>
Subject: RE: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
 representors



>-----Original Message-----
>From: Michal Swiatkowski <michal.swiatkowski@...ux.intel.com>
>Sent: Friday, July 5, 2024 2:04 PM
>To: Geethasowjanya Akula <gakula@...vell.com>
>Cc: netdev@...r.kernel.org; linux-kernel@...r.kernel.org; kuba@...nel.org;
>davem@...emloft.net; pabeni@...hat.com; edumazet@...gle.com; Sunil
>Kovvuri Goutham <sgoutham@...vell.com>; Subbaraya Sundeep Bhatta
><sbhatta@...vell.com>; Hariprasad Kelam <hkelam@...vell.com>
>Subject: Re: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
>representors
>
>On Thu, Jul 04, 2024 at 01: 48: 23PM +0000, Geethasowjanya Akula wrote: > > >
>>-----Original Message----- > >From: Michal Swiatkowski
><michal. swiatkowski@ linux. intel. com> > >Sent: Thursday, July 4, 2024 12: 45
>PM 
>On Thu, Jul 04, 2024 at 01:48:23PM +0000, Geethasowjanya Akula wrote:
>>
>>
>> >-----Original Message-----
>> >From: Michal Swiatkowski <michal.swiatkowski@...ux.intel.com>
>> >Sent: Thursday, July 4, 2024 12:45 PM
>> >To: Geethasowjanya Akula <gakula@...vell.com>
>> >Cc: netdev@...r.kernel.org; linux-kernel@...r.kernel.org;
>> >kuba@...nel.org; davem@...emloft.net; pabeni@...hat.com;
>> >edumazet@...gle.com; Sunil Kovvuri Goutham
><sgoutham@...vell.com>;
>> >Subbaraya Sundeep Bhatta <sbhatta@...vell.com>; Hariprasad Kelam
>> ><hkelam@...vell.com>
>> >Subject: Re: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
>> >representors
>> >
>> >On Wed, Jul 03, 2024 at 02: 34: 03PM +0000, Geethasowjanya Akula
>> >wrote: > >
>> >> >-----Original Message----- > >From: Michal Swiatkowski
>> ><michal. swiatkowski@ linux. intel. com> > >Sent: Wednesday, July 3,
>> >2024 4: 14 PM On Wed, Jul 03, 2024 at 02:34:03PM +0000,
>> >Geethasowjanya Akula wrote:
>> >>
>> >>
>> >> >-----Original Message-----
>> >> >From: Michal Swiatkowski <michal.swiatkowski@...ux.intel.com>
>> >> >Sent: Wednesday, July 3, 2024 4:14 PM
>> >> >To: Geethasowjanya Akula <gakula@...vell.com>
>> >> >Cc: netdev@...r.kernel.org; linux-kernel@...r.kernel.org;
>> >> >kuba@...nel.org; davem@...emloft.net; pabeni@...hat.com;
>> >> >edumazet@...gle.com; Sunil Kovvuri Goutham
>> ><sgoutham@...vell.com>;
>> >> >Subbaraya Sundeep Bhatta <sbhatta@...vell.com>; Hariprasad Kelam
>> >> ><hkelam@...vell.com>
>> >> >Subject: [EXTERNAL] Re: [net-next PATCH v7 00/10] Introduce RVU
>> >> >representors On Fri, Jun 28, 2024 at 07:05:07PM +0530, Geetha
>> >> >sowjanya
>> >wrote:
>> >> >> This series adds representor support for each rvu devices.
>> >> >> When switchdev mode is enabled, representor netdev is registered
>> >> >> for each rvu device. In implementation of representor model, one
>> >> >> NIX HW LF with multiple SQ and RQ is reserved, where each RQ and
>> >> >> SQ of the LF are mapped to a representor. A loopback channel is
>> >> >> reserved to support packet path between representors and VFs.
>> >> >> CN10K silicon supports 2 types of MACs, RPM and SDP. This patch
>> >> >> set adds representor support for both RPM and SDP MAC interfaces.
>> >> >>
>> >> >> - Patch 1: Refactors and exports the shared service functions.
>> >> >> - Patch 2: Implements basic representor driver.
>> >> >> - Patch 3: Add devlink support to create representor netdevs that
>> >> >>   can be used to manage VFs.
>> >> >> - Patch 4: Implements basec netdev_ndo_ops.
>> >> >> - Patch 5: Installs tcam rules to route packets between representor and
>> >> >> 	   VFs.
>> >> >> - Patch 6: Enables fetching VF stats via representor interface
>> >> >> - Patch 7: Adds support to sync link state between representors and VFs
>.
>> >> >> - Patch 8: Enables configuring VF MTU via representor netdevs.
>> >> >> - Patch 9: Add representors for sdp MAC.
>> >> >> - Patch 10: Add devlink port support.
>> >> >>
>> >> >> Command to create VF representor #devlink dev eswitch set
>> >> >> pci/0002:1c:00.0 mode switchdev VF representors are created for
>> >> >> each VF when switch mode is set switchdev on representor PCI
>> >> >> device
>> >> >
>> >> >Does it mean that VFs needs to be created before going to
>> >> >switchdev mode? (in legacy mode). Keep in mind that in both
>> >> >mellanox and ice driver assume that VFs are created after chaning
>> >> >mode to switchdev (mode can't be changed if VFs).
>> >> No. RVU representor driver implementation is similar to mellanox
>> >> and ice
>> >drivers.
>> >> It assumes that VF gets created only after switchdev mode is enabled.
>> >> Sorry, if above commit description is confusing. Will rewrite it.
>> >>
>> >
>> >Ok, but why the rvu_rep_create() is called in switching mode to
>> >switchdev function? In this function you are creating netdevs, only
>> >for PF representor? It looks like it doesn't called from other
>> >context in this patchset, so where the port representor netdevs for VFs are
>created?
>> >
>> RVU representors for PF/VFs are created when switchdev mode is set, similar
>to the bnxt and nfp drivers.
>> rvu_rep_create() will create representors based on rep_cnt (which
>> include both PFs and VFs count)
>>
>
>Sorry, I don't understand now. You wrote in previous message that VFs should
>be created after switchdev mode is enabled. Now you are writting that they are
>created based on rep_cnt (so assuming VFs have been created before).
>
>What is the correct order?
># echo 1 x > sriov_numvfs
># devlink dev eswitch set pci/0000:ca:00.0 mode switchdev or # devlink dev
>eswitch set pci/0000:ca:00.0 mode switchdev # echo 1 x > sriov_numvfs
>
>or you can create VFs before and after? From looking at the drivers code it
>looks like driver bnxt supports that.
Actually, we support VF creation before and after representor creation. 
Representor will not able send or receive any traffic untill corresponding representee
is created.
>
>I am not familiar with nfp and bnxt driver but based on code in current kernel
>version it looks like nfp creates representor when switching mode only for
>physical and PF types, VF representor type is created when the VF is created.
>Bnxt support VF representor creation before and after switching mode. In ice
>port representors are created during VFs creation only.
>
>I am not against your solution, only pointing that it can lead to some problems.
>
Thanks for sharing the information.

Geetha.

>Thanks,
>Michal
>
>> Thanks,
>> Geetha.
>>
>> >Thanks,
>> >Michal
>> >
>> >> >
>> >> >Different order can be problematic. For example (AFAIK) kubernetes
>> >> >scripts for switchdev assume that first is switching to switchdev
>> >> >and VFs creation is done after that.
>> >> >
>> >> >Thanks,
>> >> >Michal
>> >> >
>> >> >> # devlink dev eswitch set pci/0002:1c:00.0  mode switchdev # ip
>> >> >> link show
>> >> >> 25: r0p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>> >mode
>> >> >DEFAULT group default qlen 1000
>> >> >>     link/ether 32:0f:0f:f0:60:f1 brd ff:ff:ff:ff:ff:ff
>> >> >> 26: r1p1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN
>> >mode
>> >> >DEFAULT group default qlen 1000
>> >> >>     link/ether 3e:5d:9a:4d:e7:7b brd ff:ff:ff:ff:ff:ff
>> >> >>
>> >> >> #devlink dev
>> >> >> pci/0002:01:00.0
>> >> >> pci/0002:02:00.0
>> >> >> pci/0002:03:00.0
>> >> >> pci/0002:04:00.0
>> >> >> pci/0002:05:00.0
>> >> >> pci/0002:06:00.0
>> >> >> pci/0002:07:00.0
>> >> >>
>> >> >> ~# devlink port
>> >> >> pci/0002:1c:00.0/0: type eth netdev r0p1v0 flavour pcipf
>> >> >> controller
>> >> >> 0 pfnum 1 vfnum 0 external false splittable false
>> >> >> pci/0002:1c:00.0/1: type eth netdev r1p1v1 flavour pcivf
>> >> >> controller
>> >> >> 0 pfnum 1 vfnum 1 external false splittable false
>> >> >> pci/0002:1c:00.0/2: type eth netdev r2p1v2 flavour pcivf
>> >> >> controller
>> >> >> 0 pfnum 1 vfnum 2 external false splittable false
>> >> >> pci/0002:1c:00.0/3: type eth netdev r3p1v3 flavour pcivf
>> >> >> controller
>> >> >> 0 pfnum 1 vfnum 3 external false splittable false
>> >> >>
>> >> >> -----------
>> >> >> v1-v2:
>> >> >>  -Fixed build warnings.
>> >> >>  -Address review comments provided by "Kalesh Anakkur Purayil".
>> >> >>
>> >> >> v2-v3:
>> >> >>  - Used extack for error messages.
>> >> >>  - As suggested reworked commit messages.
>> >> >>  - Fixed sparse warning.
>> >> >>
>> >> >> v3-v4:
>> >> >>  - Patch 2 & 3: Fixed coccinelle reported warnings.
>> >> >>  - Patch 10: Added devlink port support.
>> >> >>
>> >> >> v4-v5:
>> >> >>   - Patch 3: Removed devm_* usage in rvu_rep_create()
>> >> >>   - Patch 3: Fixed build warnings.
>> >> >>
>> >> >> v5-v6:
>> >> >>   - Addressed review comments provided by "Simon Horman".
>> >> >>   - Added review tag.
>> >> >>
>> >> >> v6-v7:
>> >> >>   - Rebased on top net-next branch.
>> >> >>
>> >> >> Geetha sowjanya (10):
>> >> >>   octeontx2-pf: Refactoring RVU driver
>> >> >>   octeontx2-pf: RVU representor driver
>> >> >>   octeontx2-pf: Create representor netdev
>> >> >>   octeontx2-pf: Add basic net_device_ops
>> >> >>   octeontx2-af: Add packet path between representor and VF
>> >> >>   octeontx2-pf: Get VF stats via representor
>> >> >>   octeontx2-pf: Add support to sync link state between representor and
>> >> >>     VFs
>> >> >>   octeontx2-pf: Configure VF mtu via representor
>> >> >>   octeontx2-pf: Add representors for sdp MAC
>> >> >>   octeontx2-pf: Add devlink port support
>> >> >>
>> >> >>  .../net/ethernet/marvell/octeontx2/Kconfig    |   8 +
>> >> >>  .../ethernet/marvell/octeontx2/af/Makefile    |   3 +-
>> >> >>  .../ethernet/marvell/octeontx2/af/common.h    |   2 +
>> >> >>  .../net/ethernet/marvell/octeontx2/af/mbox.h  |  74 ++
>> >> >>  .../net/ethernet/marvell/octeontx2/af/npc.h   |   1 +
>> >> >>  .../net/ethernet/marvell/octeontx2/af/rvu.c   |  11 +
>> >> >>  .../net/ethernet/marvell/octeontx2/af/rvu.h   |  30 +-
>> >> >>  .../marvell/octeontx2/af/rvu_debugfs.c        |  27 -
>> >> >>  .../marvell/octeontx2/af/rvu_devlink.c        |   6 +
>> >> >>  .../ethernet/marvell/octeontx2/af/rvu_nix.c   |  81 ++-
>> >> >>  .../marvell/octeontx2/af/rvu_npc_fs.c         |   5 +
>> >> >>  .../ethernet/marvell/octeontx2/af/rvu_reg.h   |   4 +
>> >> >>  .../ethernet/marvell/octeontx2/af/rvu_rep.c   | 464 ++++++++++++
>> >> >>  .../marvell/octeontx2/af/rvu_struct.h         |  26 +
>> >> >>  .../marvell/octeontx2/af/rvu_switch.c         |  20 +-
>> >> >>  .../ethernet/marvell/octeontx2/nic/Makefile   |   2 +
>> >> >>  .../ethernet/marvell/octeontx2/nic/cn10k.c    |   4 +-
>> >> >>  .../ethernet/marvell/octeontx2/nic/cn10k.h    |   2 +-
>> >> >>  .../marvell/octeontx2/nic/otx2_common.c       |  56 +-
>> >> >>  .../marvell/octeontx2/nic/otx2_common.h       |  84 ++-
>> >> >>  .../marvell/octeontx2/nic/otx2_devlink.c      |  49 ++
>> >> >>  .../ethernet/marvell/octeontx2/nic/otx2_pf.c  | 305 +++++---
>> >> >>  .../marvell/octeontx2/nic/otx2_txrx.c         |  38 +-
>> >> >>  .../marvell/octeontx2/nic/otx2_txrx.h         |   3 +-
>> >> >>  .../ethernet/marvell/octeontx2/nic/otx2_vf.c  |  19 +-
>> >> >> .../net/ethernet/marvell/octeontx2/nic/rep.c  | 684
>> >> >> ++++++++++++++++++ .../net/ethernet/marvell/octeontx2/nic/rep.h
>> >> >> ++++++++++++++++++ |
>> >> >> 53 ++
>> >> >>  27 files changed, 1834 insertions(+), 227 deletions(-)  create
>> >> >> mode
>> >> >> 100644 drivers/net/ethernet/marvell/octeontx2/af/rvu_rep.c
>> >> >>  create mode 100644
>> >> >> drivers/net/ethernet/marvell/octeontx2/nic/rep.c
>> >> >>  create mode 100644
>> >> >> drivers/net/ethernet/marvell/octeontx2/nic/rep.h
>> >> >>
>> >> >> --
>> >> >> 2.25.1
>> >> >>

Powered by blists - more mailing lists