Message-ID: <cf30ce2e-ab70-4bbe-82ab-d687c2ea2efc@intel.com>
Date: Tue, 9 Apr 2024 14:20:23 +0200
From: Wojciech Drewek <wojciech.drewek@...el.com>
To: Andrew Lunn <andrew@...n.ch>
CC: <netdev@...r.kernel.org>, <idosch@...dia.com>, <edumazet@...gle.com>,
	<marcin.szycik@...ux.intel.com>, <anthony.l.nguyen@...el.com>,
	<kuba@...nel.org>, <intel-wired-lan@...ts.osuosl.org>, <pabeni@...hat.com>,
	<przemyslaw.kitszel@...el.com>
Subject: Re: [Intel-wired-lan] [PATCH net-next 0/3] ethtool: Max power support



On 04.04.2024 15:53, Andrew Lunn wrote:
> On Thu, Apr 04, 2024 at 02:45:43PM +0200, Wojciech Drewek wrote:
>>
>>
>> On 03.04.2024 15:49, Andrew Lunn wrote:
>>>>> $ ethtool --set-module enp1s0f0np0 power-max-set 4000
>>>>>
>>>>> actually talk to the SFP module and tell it the maximum power it can
>>>>> consume. So in this case, it is not the cage, but the module?
>>>>
>>>> It does not work that way in ice example.
>>>>>
>>>>> Or is it talking to some entity which is managing the overall power
>>>>> consumption of a number of cages, and asking it to allocate a maximum
>>>>> of 4W to this cage. It might return an error message saying there is
>>>>> no power budget left?
>>>>
>>>> That's right, we talk to firmware to set those restrictions.
>>>> In the ice implementation, the driver is responsible for checking if the
>>>> overall board budget is not exceeded.
>>>
>>> So i can get the board to agree that the cage can supply 3W to the
>>> module, but how do i then tell the module this?
>>
>> I'd assume it is not possible, if the module consumes more power
>> than maximum than the link will not come up and error will be printed.
> 
> Take a look at the Linux SFP driver. In sfp_probe() is reads the DT
> property maximum-power-milliwatt:
> 
> https://elixir.bootlin.com/linux/latest/source/drivers/net/phy/sfp.c#L3030
> 
> When the module is inserted and probed, the modules power capabilities
> are read from the EEPROM:
> 
> https://elixir.bootlin.com/linux/latest/source/drivers/net/phy/sfp.c#L2320
> 
> https://elixir.bootlin.com/linux/latest/source/drivers/net/phy/sfp.c#L1929
> 
> The code looks to see what conformance level the module has, so to
> know if it even supports different power levels, and the registers
> needed have been implemented.
> 
> Later, the SFP state machine will transition the module to higher
> power:
> https://elixir.bootlin.com/linux/latest/source/drivers/net/phy/sfp.c#L1995
> 
> by writing a register in the SFP.
> 
>>> I would also suggest you don't focus too much on ICE. I find it better
>>> to think about an abstract system. A board with a power supply to a
>>> number of SFP cages, and some cages have modules in them. What does
>>> the kAPI look like, the use cases for this abstract system.
>>
>> My design for this API is to have an option to get and set maximum
>> power that the module in the cage can consume. It's not about modifying
>> module's power consumption, it's about setting restrictions for it.
>>
>> The use case is to let the user change maximum power in the given cage
>> (so he can plug in the module with higher power consumption). Before that
>> he will lower maximum power in different cage. Thanks to that the overall
>> budget for the board won't be exceeded. Does it make sense for the abstract
>> system you described?
> 
> So there are a few different phases here. The standard says the module
> start up in low power mode.
> 
> Something needs to enumerate what the module actually supports in
> terms of different power modes.

So, this is something that would require new netlink attribute.
>From what you described, SFP driver is capable of reading module's
power capabilities so it could be implemented there.

> 
> Something then needs to determine if the board/cage can support higher
> power operation, that there is sufficient power budged. Budget then
> needs to be allocated to the cage.

This is something my current design supports I think. Using
ETHTOOL_A_MODULE_MAX_POWER_SET user can get what cage supports
and change it.

> 
> Lastly, the module needs to be told it can go to a higher power level.

This could be done using ethtool_module_power_mode_policy I think.

> 
> I would say the current Linux SFP code is overly simple, by design. It
> supports the concept of cage supplied by a dedicated regulator. There
> is no sharing of power across a number of cages. Hence it just ups the
> power if the board indicates the power is available and the module
> support a higher power level.
> 
> However, you have a more complex setup, shared power supplies,
> etc. The policy of what modules gets how much power should come from
> user space. So we need user space APIs for this, and a clear
> understanding of how they work. Please could you describe how i would
> use ethtool to go through these phases. And how do i down grade a
> modules power consumption to make more power available to another
> module.

I described how I see it for each phase above. 
Downgrading module's power consumption is not possible with my initial
design. You can downgrade max power for the cage and if the module
plugged in to it consumes more power than error will be printed and
link will not be established.
In the end it looks like we would need new netlink attribute to support
modification of module's power consumption.

> 
> 	Andrew
> 
> 

Powered by blists - more mailing lists