netdev - RE: [PATCH v2 05/23] ice: Add devlink params support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <BY5PR12MB43226856B588B5666C463A5CDC619@BY5PR12MB4322.namprd12.prod.outlook.com>
Date:   Fri, 26 Mar 2021 05:42:26 +0000
From:   Parav Pandit <parav@...dia.com>
To:     "Saleem, Shiraz" <shiraz.saleem@...el.com>,
        "dledford@...hat.com" <dledford@...hat.com>,
        Jason Gunthorpe <jgg@...dia.com>,
        "kuba@...nel.org" <kuba@...nel.org>,
        "davem@...emloft.net" <davem@...emloft.net>
CC:     "linux-rdma@...r.kernel.org" <linux-rdma@...r.kernel.org>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "Ertman, David M" <david.m.ertman@...el.com>,
        "Nguyen, Anthony L" <anthony.l.nguyen@...el.com>
Subject: RE: [PATCH v2 05/23] ice: Add devlink params support



> From: Saleem, Shiraz <shiraz.saleem@...el.com>
> Sent: Friday, March 26, 2021 1:40 AM
> > Subject: RE: [PATCH v2 05/23] ice: Add devlink params support

[..]
> >
> > Resources are better represented as devlink resource.
> > Such as,
> >
> > $ devlink resource set pci/0000:06:00.0 /rdma/max_qps 16384 $ devlink
> > resource set pci/0000:06:00.0 /rdma/max_cqs 8192 $ devlink resource
> > set pci/0000:06:00.0 /rdma/max_mrs 16384
> >
> 
> Hi Parav - Thank you for the feedback.
> 
> Maybe I am missing something but I see that a devlink hot reload is required
> to enforce the update?
It isn't mandatory to reload, but yes either reload or driver unbind/bind is needed as you suggested below.

> There isn't really a de-init required of PCI driver entities in this case for this
> rdma param.
> But only an unplug, plug of the auxdev with new value. Intuitively it feels
> more runtime-ish.
> 
Driver unbind/bind to reflect new limits is ok for cases where it is not time sensitive.
For mlx5 use cases, user expects device to be provisioned in < few msecs.
And driver unbind/bind are sub-optimal to achieve it from time and memory wise.
So mlx5 driver prefers to stay away from driver unbind/bind steps.
So I am working on series to not create class aux devices by default for SFs.
Rather to create them on reload.
Something like, 
$ devlink dev param set pci/0000:06:00.0 name pcisf_classes value false cmode driverinit
$ devlink dev class set auxiliary/mlx5_core.sf.4 rdma true
$ devlink resource set auxiliary/mlx5_core.sf.4 path rdma/max_qps size 200000
$ devlink dev reload auxiliary/mlx5_core.sf.4
This last command will create the mlx5_core.rdma.4 aux device and when its bound to driver, it will create IB device with right max_qp.
So rdma device is created only once, instead of twice using unbind/bind sequence.

This may not be possible for the PF/VF device due to backward compatibility and their different usage in system.

> There is also a device powerof2 requirement on the maxqp which I don't see
> enforceable as it stands.
>
Right, but similar to size_params.size_max, size_granularity, I believe size_params can be extended for alignment restriction.

> This is not super-critical for the initial submission but a nice to have. But I do
> want to brainstorm options..
> 
If max_qp is truly a dynamic value that doesn't require device recreation,
extending existing rdma resource command seems more useful to end user than doing unbind/bind.