Message-ID: <c819700a-97b6-9993-491b-599b82842dc2@linaro.org>
Date:   Fri, 2 Apr 2021 13:10:51 +0200
From:   Daniel Lezcano <daniel.lezcano@...aro.org>
To:     Greg KH <gregkh@...uxfoundation.org>
Cc:     linux-pm@...r.kernel.org, linux-kernel@...r.kernel.org,
        lukasz.luba@....com, rafael@...nel.org,
        Ram Chandrasekar <rkumbako@...eaurora.org>
Subject: Re: [PATCH v6 2/7] powercap/drivers/dtpm: Create a registering system

On 02/04/2021 10:02, Greg KH wrote:
> On Fri, Apr 02, 2021 at 12:08:49AM +0200, Daniel Lezcano wrote:
>>
>> Hi Greg,
>>
>> On 01/04/2021 21:28, Greg KH wrote:
>>> On Thu, Apr 01, 2021 at 08:36:49PM +0200, Daniel Lezcano wrote:
>>>> A SoC can be differently structured depending on the platform and the
>>>> kernel can not be aware of all the combinations, as well as the
>>>> specific tweaks for a particular board.
>>>>
>>>> The creation of the hierarchy must be delegated to userspace.
>>>
>>> Why?  Isn't this what DT is for?
>>
>> I've always been told the DT describes the hardware. Here we are more
>> describing a configuration, that is the reason why I've let the
>> userspace to handle that through configfs.
> 
> DT does describe how the hardware configuration is to be used.  You are
> saying that the hardware description does not work and somehow you need
> a magic userspace tool to reconfigure things instead?
> 
>>> What "userspace tool" is going to be created to manage all of this?
>>> Pointers to that codebase?
>>
>> You are certainly aware of most of it but let me give a bit more of context.
> 
> No, I am not aware of it at all, thanks :)
> 
>> The thermal framework has cooling devices which export their 'state', a
>> representation of their performance level, in sysfs. Unfortunately that
>> gives access from the user space to act on the performance as a power
>> limiter in total conflict with the in-kernel thermal governor decisions.
> 
> Why not remove that conflict?

Because the cooling device state should have not be exported its state
in a writable way, that is a weakness different applications are
abusing. Moreover, the thermal framework is not designed for that. It is
not the purpose of the cooling device to be managed as a power limiter
by the userspace.

The powercap framework is there for that.

There are devices registering themselves as a cooling device while there
is no sensor and governor for them just for the sake of acting on their
power via opaque integer values.


>> That is done from thermal daemon the different SoC vendors tweak for
>> their platform. Depending on the application running and identified as a
>> scenario, the daemon acts proactively on the different cooling devices
>> to ensure a skin temperature which is far below the thermal limit of the
>> components.
> 
> So userspace is going to try to manage power settings in order to keep
> thermal values under control?  Has no one learned from our past mistakes
> when we used to have to do this 10 years ago and it turned out so bad
> that it was just baked into the hardware instead?

I agree in a desktop/server environment, that is true but on the
embedded world, things are a bit different because we are on battery,
the cooling devices are passive and the device is carried. This is why
there are different level of thermal control:

 - In the hardware / firmware to protect physically the silicon

 - In the kernel, to protect from a hard reset or reboot. Usually each
of them is represented with a thermal zone (and its sensor) and a
governor. There is a 1:1 relationship. IOW, a governor manage a thermal
zone without any knowledge of the other thermal zones temperature, just
the one it is monitoring and acts on the power (with different
techniques) when the temperature threshold is reached. Its action is
local and it is after crossing a certain temperature for this component.

 - In the userspace, the logic will get notified about which application
is running and can set the power limitation on different devices knowing
the performances are enough for that particular application, that saves
energy and consequently reduce the temperature. Its action is on
multiple places and happens far below the thermal limits. On the other
side, the skin temperature is a particular sensor placed on the
dissipation path and its temperature must be below 45°C (30 minutes skin
contact at 45°C causes a 1st degree burn). It is the results of the
dissipation of all the devices, the logic in userspace can know which
devices to act on to have the overall power dissipation low enough to
stay below this 45°C (eg. reduce camera resolution).  So here userspace
deals with performance, temperature, application profile, etc ... and
abuse the cooling device.

I'm not sure in the ARM/ARM64 embedded ecosystem, creating a complex
hardware mechanism, supported by a firmware is really possible.

In a desktop/server environment, we do not care about having this skin
temperature (and battery savings) aspects.

> {sigh}
> 
>> This usage of the cooling devices hijacked the real purpose of the
>> thermal framework which is to protect the silicon. Nobody to blame,
>> there is no alternative for userspace.
> 
> Userspace should not care.
> 
>> The use case falls under the power limitation framework prerogative and
>> that is the reason why we provided a new framework to limit the power
>> based on the powercap framework. The thermal daemon can then use it and
>> stop abusing the thermal framework.
>>
>> This DTPM framework allows to read the power consumption and set a power
>> limit to a device.
>>
>> While the powercap simple backend gives a single device entry, DTPM
>> aggregates the different devices power by summing their power and their
>> limits. The tree representation of the different DTPM nodes describe how
>> their limits are set and how the power is computed along the different
>> devices.
> 
> That's all great, doing this in-kernel is fine, it's now the "userspace
> must set this up properly that I'm objecting to as no one will know how
> to do this.
> 
>> For more info, we did a presentation at ELC [1] and Linux PM
>> microconference [2] and there is an article talking about it [3].
>>
>>
>> To answer your questions, there is a SoC vendor thermal daemon using
>> DTPM and there is a tool created to watch the thermal framework and read
>> the DTPM values, it is available at [4]. It is currently under
>> development with the goal of doing power rebalancing / capping across
>> the different nodes when there is a violation of the parent's power limit.
> 
> Crazy ideas aside, your implementation of this is my main objection
> here.  You are creating a user/kernel api that you will have to support
> for 20+ years, without a real userspace user just yet (from what I can
> tell).  That's rough, and is going to mean that this gets messy over
> time.

I'm not sure to understand, the API already exists since v3.3, it is the
powercap and DTPM is its backend. AFAICT, there are already users of it
except they create their own way to build the hierarchy today.

> Also there's the whole "tying sysfs to configfs" mess and reference
> counting that I object to as well...

Ok, I think my description was bad but I agree with the fact that we
must get rid of any adherence between sysfs and configfs. Thanks for
reporting the issue.

[ ... ]

>> The configfs does not represent the layout of the sensors or the floor
>> plan of the devices but only the constraints we want to tie together.
>>
>> That is the reason why I think using configfs instead of OF is more
>> adequate and flexible as userspace deals with the power numbers.
>> Moreover, we won't be facing issues with devices initialization priority
>> when the daemon starts.
>>
>> I thought we can add OF later, when the framework has more users and
>> more devices. The configfs and OF can certainly co-exist or be mutually
>> exclusive via the Kconfig option.
> 
> Kconfig options are not ok for this, you want to build a kernel that
> works everywhere.
> 
> If you want to make virtual things, that's fine, but again, your tying
> of sysfs to configfs in odd ways, along with the reference counting
> bugs/nightmare the current implementation is creating, is a non-starter
> for me at the moment, sorry.

I did not realize this reference counting issue. Thanks for spotting it.

The points you raised are IMO solvable and I can send a new version but
with all the comments I'm a bit lost with the approach with configfs.

Do you think it is preferable to use OF instead ?

Thanks again for your time

  -- Daniel

-- 
<http://www.linaro.org/> Linaro.org │ Open source software for ARM SoCs

Follow Linaro:  <http://www.facebook.com/pages/Linaro> Facebook |
<http://twitter.com/#!/linaroorg> Twitter |
<http://www.linaro.org/linaro-blog/> Blog

Powered by blists - more mailing lists