Message-ID: <Zc8YgDOf6jKIHNsF@nanopsycho>
Date: Fri, 16 Feb 2024 09:10:40 +0100
From: Jiri Pirko <jiri@...nulli.us>
To: Jacob Keller <jacob.e.keller@...el.com>
Cc: Jakub Kicinski <kuba@...nel.org>, William Tu <witu@...dia.com>,
	bodong@...dia.com, jiri@...dia.com, netdev@...r.kernel.org,
	saeedm@...dia.com,
	"aleksander.lobakin@...el.com" <aleksander.lobakin@...el.com>
Subject: Re: [RFC PATCH v3 net-next] Documentation: devlink: Add devlink-sd

Thu, Feb 15, 2024 at 06:41:31PM CET, jacob.e.keller@...el.com wrote:
>
>
>On 2/15/2024 5:19 AM, Jiri Pirko wrote:
>> Fri, Feb 09, 2024 at 02:26:33AM CET, kuba@...nel.org wrote:
>>> On Fri, 2 Feb 2024 08:46:56 +0100 Jiri Pirko wrote:
>>>> Fri, Feb 02, 2024 at 05:00:41AM CET, kuba@...nel.org wrote:
>>>>> On Thu, 1 Feb 2024 11:13:57 +0100 Jiri Pirko wrote:  
>>>>>> Wait a sec.  
>>>>>
>>>>> No, you wait a sec ;) Why do you think this belongs to devlink?
>>>>> Two months ago you were complaining bitterly when people were
>>>>> considering using devlink rate to control per-queue shapers.
>>>>> And now it's fine to add queues as a concept to devlink?  
>>>>
>>>> Do you have a better suggestion how to model common pool object for
>>>> multiple netdevices? This is the reason why devlink was introduced to
>>>> provide a platform for common/shared things for a device that contains
>>>> multiple netdevs/ports/whatever. But I may be missing something here,
>>>> for sure.
>>>
>>> devlink just seems like the lowest common denominator, but the moment
>>> we start talking about multi-PF devices it also gets wobbly :(
>> 
>> You mean you see real to have a multi-PF device that allows to share the
>> pools between the PFs? If, in theory, that exists, could this just be a
>> limitation perhaps?
>> 
>> 
>
>I don't know offhand if we have a device which can share pools
>specifically, but we do have multi-PF devices which have a lot of shared
>resources. However, due to the multi-PF PCIe design. I looked into ways
>to get a single devlink across the devices.. but ultimately got stymied
>and gave up.
>
>This left us with accepting the limitation that each PF gets its own
>devlink and can't really communicate with other PFs.
>
>The existing solution has just been to partition the shared resources
>evenly across PFs, typically via firmware. No flexibility.
>
>I do think the best solution here would be to figure out a generic way
>to tie multiple functions into a single devlink representing the device.
>Then each function gets the set of devlink_port objects associated to
>it. I'm not entirely sure how that would work. We could hack something
>together with auxbus.. but thats pretty ugly. Some sort of orchestration
>in the PCI layer that could identify when a device wants to have some
>sort of "parent" driver which loads once and has ties to each of the
>function drivers would be ideal.

Hmm, dpll does this. You basically share dpll device instance in between
multiple pci functions. The same concept could be done for devlink. We
have to figure out the backward compatibility. User now expects the
instances per-pf.


>
>Then this parent driver could register devlink, and each function driver
>could connect to it and allocate ports and function-specific resources.
>
>Alternatively a design which loads a single driver that maintains
>references to each function could work but that requires a significant
>change to the entire driver design and is unlikely to be done for
>existing drivers...
>
>This is obviously a bit more of a tangential problem to this series.

Powered by blists - more mailing lists