Message-ID: <ffd2df1e-71fa-ada1-ed95-69612f3b5072@linaro.org>
Date:   Mon, 12 Jun 2023 19:03:04 +0200
From:   Konrad Dybcio <konrad.dybcio@...aro.org>
To:     Stephan Gerhold <stephan@...hold.net>
Cc:     Andy Gross <agross@...nel.org>,
        Bjorn Andersson <andersson@...nel.org>,
        Michael Turquette <mturquette@...libre.com>,
        Stephen Boyd <sboyd@...nel.org>,
        Georgi Djakov <djakov@...nel.org>,
        Leo Yan <leo.yan@...aro.org>,
        Evan Green <evgreen@...omium.org>,
        Marijn Suijten <marijn.suijten@...ainline.org>,
        linux-arm-msm@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-clk@...r.kernel.org, linux-pm@...r.kernel.org
Subject: Re: [PATCH v2 04/22] clk: qcom: smd-rpm: Export clock scaling
 availability



On 12.06.2023 14:51, Konrad Dybcio wrote:
> On 11.06.2023 11:20, Stephan Gerhold wrote:
>> On Sat, Jun 10, 2023 at 09:39:00PM +0200, Konrad Dybcio wrote:
>>> On 10.06.2023 21:25, Stephan Gerhold wrote:
>>>> On Sat, Jun 10, 2023 at 08:53:05PM +0200, Konrad Dybcio wrote:
>>>>> On 10.06.2023 14:15, Konrad Dybcio wrote:
>>>>>> On 10.06.2023 13:35, Stephan Gerhold wrote:
>>>>>>> On Fri, Jun 09, 2023 at 10:19:09PM +0200, Konrad Dybcio wrote:
>>>>>>>> Before we issue a call to RPM through clk_smd_rpm_enable_scaling() the
>>>>>>>> clock rate requests will not be commited in hardware. This poses a
>>>>>>>> race threat since we're accessing the bus clocks directly from within
>>>>>>>> the interconnect framework.
>>>>>>>>
>>>>>>>> Add a marker to indicate that we're good to go with sending new requests
>>>>>>>> and export it so that it can be referenced from icc.
>>>>>>>>
>>>>>>>> Signed-off-by: Konrad Dybcio <konrad.dybcio@...aro.org>
>>>>>>>> ---
>>>>>>>>  drivers/clk/qcom/clk-smd-rpm.c   | 9 +++++++++
>>>>>>>>  include/linux/soc/qcom/smd-rpm.h | 2 ++
>>>>>>>>  2 files changed, 11 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> index 937cb1515968..482fe30ee6f0 100644
>>>>>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>>>>>> @@ -151,6 +151,7 @@
>>>>>>>>  #define to_clk_smd_rpm(_hw) container_of(_hw, struct clk_smd_rpm, hw)
>>>>>>>>  
>>>>>>>>  static struct qcom_smd_rpm *rpmcc_smd_rpm;
>>>>>>>> +static bool smd_rpm_clk_scaling;
>>>>>>>>  
>>>>>>>>  struct clk_smd_rpm {
>>>>>>>>  	const int rpm_res_type;
>>>>>>>> @@ -385,6 +386,12 @@ static unsigned long clk_smd_rpm_recalc_rate(struct clk_hw *hw,
>>>>>>>>  	return r->rate;
>>>>>>>>  }
>>>>>>>>  
>>>>>>>> +bool qcom_smd_rpm_scaling_available(void)
>>>>>>>> +{
>>>>>>>> +	return smd_rpm_clk_scaling;
>>>>>>>> +}
>>>>>>>> +EXPORT_SYMBOL_GPL(qcom_smd_rpm_scaling_available);
>>>>>>>> +
>>>>>>>>  static int clk_smd_rpm_enable_scaling(void)
>>>>>>>>  {
>>>>>>>>  	int ret;
>>>>>>>> @@ -410,6 +417,8 @@ static int clk_smd_rpm_enable_scaling(void)
>>>>>>>>  		return ret;
>>>>>>>>  	}
>>>>>>>>  
>>>>>>>> +	smd_rpm_clk_scaling = true;
>>>>>>>> +
>>>>>>>
>>>>>>> If you move the platform_device_register_data(&rpdev->dev,
>>>>>>> "icc_smd_rpm", ...) from drivers/soc/qcom/smd-rpm.c to here you can
>>>>>>> avoid the race completely and drop this API. I think that would be
>>>>>>> cleaner. And it will likely probe much faster because probe deferral
>>>>>>> is slow. :)
>>>>>> Sounds like an idea.. especially since it's pretty much the only
>>>>>> dependency other than SMDRPM itself!
>>>>> It sounds great, but to not break bisecting one has to:
>>>>>
>>>>> 1. change the registration in soc/smd-rpm to store rpm ptr in driver
>>>>>    data, in addition to parent driver data
>>>>>
>>>>> 2. change icc/smd-rpm to use the device and not parent data
>>>>>
>>>>> 3. add a platform_device_register_data call in clk-smd-rpm that will
>>>>>    always fail because the device is always registered
>>>>>
>>>>> 4. remove the registration from soc/smd-rpm
>>>>>
>>>>
>>>> Logically the icc_smd_rpm device still fits better as child of
>>>> smd-rpm and not clk-smd-rpm. So I would probably just continue
>>>> registering it on the parent device from clk-smd-rpm.
>>>> Then there are no changes necessary in icc_smd_rpm.
>>>>
>>>> You could use this. Both touched files are Bjorn-maintained so should be
>>>> manageable to have it in one commit. (note: compile-tested only)
>>>>
>>>> Thanks,
>>>> Stephan
>>>>
>>>> From a2610adb2551b01e76b9de8e4cbcc89853814a8f Mon Sep 17 00:00:00 2001
>>>> From: Stephan Gerhold <stephan@...hold.net>
>>>> Date: Sat, 10 Jun 2023 21:19:48 +0200
>>>> Subject: [PATCH] soc: qcom: smd-rpm: Move icc_smd_rpm registration to
>>>>  clk-smd-rpm
>>>>
>>>> icc_smd_rpm will do bus clock votes itself rather than taking the
>>>> unnecessary detour through the clock subsystem. However, it can only
>>>> do that after the clocks have been handed off and scaling has been
>>>> enabled in the RPM in clk-smd-rpm.
>>>>
>>>> Move the icc_smd_rpm registration from smd-rpm.c to clk-smd-rpm.c
>>>> to avoid any possible races. icc_smd_rpm gets the driver data from
>>>> the smd-rpm device, so still register the platform device on the
>>>> smd-rpm parent device.
>>>>
>>>> Signed-off-by: Stephan Gerhold <stephan@...hold.net>
>>>> ---
>>> Generally it looks good.. I'll give it a spin next week. One
>>> thing below.
>>>
>>>>  drivers/clk/qcom/clk-smd-rpm.c | 21 +++++++++++++++++++++
>>>>  drivers/soc/qcom/smd-rpm.c     | 23 +----------------------
>>>>  2 files changed, 22 insertions(+), 22 deletions(-)
>>>>
>>>> diff --git a/drivers/clk/qcom/clk-smd-rpm.c b/drivers/clk/qcom/clk-smd-rpm.c
>>>> index e4de74b68797..91adb16889b3 100644
>>>> --- a/drivers/clk/qcom/clk-smd-rpm.c
>>>> +++ b/drivers/clk/qcom/clk-smd-rpm.c
>>>> @@ -1302,12 +1302,20 @@ static struct clk_hw *qcom_smdrpm_clk_hw_get(struct of_phandle_args *clkspec,
>>>>  	return desc->clks[idx] ? &desc->clks[idx]->hw : ERR_PTR(-ENOENT);
>>>>  }
>>>>  
>>>> +static void rpm_smd_unregister_icc(void *data)
>>>> +{
>>>> +	struct platform_device *icc_pdev = data;
>>>> +
>>>> +	platform_device_unregister(icc_pdev);
>>>> +}
>>>> +
>>>>  static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>>  {
>>>>  	int ret;
>>>>  	size_t num_clks, i;
>>>>  	struct clk_smd_rpm **rpm_smd_clks;
>>>>  	const struct rpm_smd_clk_desc *desc;
>>>> +	struct platform_device *icc_pdev;
>>>>  
>>>>  	rpmcc_smd_rpm = dev_get_drvdata(pdev->dev.parent);
>>>>  	if (!rpmcc_smd_rpm) {
>>>> @@ -1357,6 +1365,19 @@ static int rpm_smd_clk_probe(struct platform_device *pdev)
>>>>  	if (ret)
>>>>  		goto err;
>>>>  
>>>> +	icc_pdev = platform_device_register_data(pdev->dev.parent,
>>>> +						 "icc_smd_rpm", -1, NULL, 0);
>>>> +	if (IS_ERR(icc_pdev)) {
>>>> +		dev_err(&pdev->dev, "Failed to register icc_smd_rpm device: %pE\n",
>>>> +			icc_pdev);
>>>> +		/* No need to unregister clocks because of this */
>>>> +	} else {
>>>> +		ret = devm_add_action_or_reset(&pdev->dev, rpm_smd_unregister_icc,
>>>> +					       icc_pdev);
>>>> +		if (ret)
>>>> +			goto err;
>>>> +	}
>>>> +
>>>>  	return 0;
>>>>  err:
>>>>  	dev_err(&pdev->dev, "Error registering SMD clock driver (%d)\n", ret);
>>>> diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
>>>> index 0c1aa809cc4e..427dd5392b82 100644
>>>> --- a/drivers/soc/qcom/smd-rpm.c
>>>> +++ b/drivers/soc/qcom/smd-rpm.c
>>>> @@ -19,7 +19,6 @@
>>>>  /**
>>>>   * struct qcom_smd_rpm - state of the rpm device driver
>>>>   * @rpm_channel:	reference to the smd channel
>>>> - * @icc:		interconnect proxy device
>>>>   * @dev:		rpm device
>>>>   * @ack:		completion for acks
>>>>   * @lock:		mutual exclusion around the send/complete pair
>>>> @@ -27,7 +26,6 @@
>>>>   */
>>>>  struct qcom_smd_rpm {
>>>>  	struct rpmsg_endpoint *rpm_channel;
>>>> -	struct platform_device *icc;
>>>>  	struct device *dev;
>>>>  
>>>>  	struct completion ack;
>>>> @@ -197,7 +195,6 @@ static int qcom_smd_rpm_callback(struct rpmsg_device *rpdev,
>>>>  static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>>  {
>>>>  	struct qcom_smd_rpm *rpm;
>>>> -	int ret;
>>>>  
>>>>  	rpm = devm_kzalloc(&rpdev->dev, sizeof(*rpm), GFP_KERNEL);
>>>>  	if (!rpm)
>>>> @@ -210,24 +207,7 @@ static int qcom_smd_rpm_probe(struct rpmsg_device *rpdev)
>>>>  	rpm->rpm_channel = rpdev->ept;
>>>>  	dev_set_drvdata(&rpdev->dev, rpm);
>>>>  
>>>> -	rpm->icc = platform_device_register_data(&rpdev->dev, "icc_smd_rpm", -1,
>>>> -						 NULL, 0);
>>>> -	if (IS_ERR(rpm->icc))
>>>> -		return PTR_ERR(rpm->icc);
>>>> -
>>>> -	ret = of_platform_populate(rpdev->dev.of_node, NULL, NULL, &rpdev->dev);
>>>> -	if (ret)
>>>> -		platform_device_unregister(rpm->icc);
>>>> -
>>>> -	return ret;
>>>> -}
>>>> -
>>>> -static void qcom_smd_rpm_remove(struct rpmsg_device *rpdev)
>>>> -{
>>>> -	struct qcom_smd_rpm *rpm = dev_get_drvdata(&rpdev->dev);
>>>> -
>>>> -	platform_device_unregister(rpm->icc);
>>>> -	of_platform_depopulate(&rpdev->dev);
>>>> +	return devm_of_platform_populate(&rpdev->dev);
>>>>  }
>>>>  
>>>>  static const struct of_device_id qcom_smd_rpm_of_match[] = {
>>>> @@ -256,7 +236,6 @@ MODULE_DEVICE_TABLE(of, qcom_smd_rpm_of_match);
>>>>  
>>>>  static struct rpmsg_driver qcom_smd_rpm_driver = {
>>>>  	.probe = qcom_smd_rpm_probe,
>>>> -	.remove = qcom_smd_rpm_remove,
>>> This reaches over the removal of the icc registration, the depopulate
>>> call should stay.
>>>
>>
>> I switched the of_platform_populate() to devm_of_platform_populate(),
>> that's why the remove callback is no longer necessary. It's a bit
>> hidden, perhaps it would be enough to add to the commit message:
>>
>> "While at it, switch the remaining of_platform_populate() call to the
>>  devm variant and remove the remove callback."
>>
>> Or maybe it should be split into two patches.
> Gave it a spin, I think it ends up being worse if an IPA rpm clock is
> consumed by one of the icc providers, and that's sadly the case
> for almost all platforms (or supposed to be).. :/ Only qcm2290 doesn't
> seem to care if we poke the network interface units with half the soc
> off :P
Actually it's more complex and that rings the same tons-of-probe-deferrals
bell that we have to get rid of.. The icc_smd_rpm driver actually does
probe faster with your patch, but that's not the case with the SoC icc
provider driver which does probe later :/

Don't think either solution is perfect, but I'm willing to take your
solution, as the one I proposed is a bit of a dead end for future
improvements.

Konrad
> 
> Konrad
>>
>> Thanks,
>> Stephan

Powered by blists - more mailing lists