linux-kernel - Re: [PATCH v5 5/5] drm/amdgpu: do not resume device in thaw for normal hibernation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <b8f0339d-0221-4f35-8db1-399471db5330@amd.com>
Date: Thu, 10 Jul 2025 14:18:21 +0800
From: "Zhang, GuoQing (Sam)" <guoqzhan@....com>
To: "Lazar, Lijo" <lijo.lazar@....com>, Mario Limonciello
	<mario.limonciello@....com>, Samuel Zhang <guoqing.zhang@....com>,
	<alexander.deucher@....com>, <christian.koenig@....com>, <rafael@...nel.org>,
	<len.brown@...el.com>, <pavel@...nel.org>, <gregkh@...uxfoundation.org>,
	<dakr@...nel.org>, <airlied@...il.com>, <simona@...ll.ch>,
	<ray.huang@....com>, <matthew.auld@...el.com>, <matthew.brost@...el.com>,
	<maarten.lankhorst@...ux.intel.com>, <mripard@...nel.org>,
	<tzimmermann@...e.de>
CC: <victor.zhao@....com>, <haijun.chang@....com>, <Qing.Ma@....com>,
	<Owen.Zhang2@....com>, <linux-pm@...r.kernel.org>,
	<linux-kernel@...r.kernel.org>, <amd-gfx@...ts.freedesktop.org>,
	<dri-devel@...ts.freedesktop.org>
Subject: Re: [PATCH v5 5/5] drm/amdgpu: do not resume device in thaw for
 normal hibernation


On 2025/7/10 12:18, Lazar, Lijo wrote:
>
> On 7/10/2025 1:20 AM, Mario Limonciello wrote:
>> On 7/9/2025 6:05 AM, Samuel Zhang wrote:
>>> For normal hibernation, GPU do not need to be resumed in thaw since it is
>>> not involved in writing the hibernation image. Skip resume in this case
>>> can reduce the hibernation time.
>>>
>>> On VM with 8 * 192GB VRAM dGPUs, 98% VRAM usage and 1.7TB system memory,
>>> this can save 50 minutes.
>>>
>>> Signed-off-by: Samuel Zhang <guoqing.zhang@....com>
>> I hand modified the patches for other changes missing from linux-next in
>> your base.
>>
>> I checked on an APU with an eDP display connected and from a VT
>> hibernate does keep the display off now so this is definitely an
>> improvement there too.
>>
>> Tested-by: Mario Limonciello <mario.limonciello@....com>


Thank you, Mario!


>>
>>> ---
>>>    drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 17 +++++++++++++++++
>>>    1 file changed, 17 insertions(+)
>>>
>>> diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/
>>> drm/amd/amdgpu/amdgpu_drv.c
>>> index 4f8632737574..b24c420983ef 100644
>>> --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>> +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
>>> @@ -2541,6 +2541,10 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
>>>        if (amdgpu_ras_intr_triggered())
>>>            return;
>>>    +    /* device maybe not resumed here, return immediately in this
>>> case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return;
>>> +
>>>        /* if we are running in a VM, make sure the device
>>>         * torn down properly on reboot/shutdown.
>>>         * unfortunately we can't detect certain
>>> @@ -2557,6 +2561,10 @@ static int amdgpu_pmops_prepare(struct device
>>> *dev)
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>>        struct amdgpu_device *adev = drm_to_adev(drm_dev);
>>>    +    /* device maybe not resumed here, return immediately in this
>>> case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return 0;
>>> +
>> Is this one right?  Don't we still want to call prepare() for all the HW
>> IP blocks?  The eviction call that happens in prepare() is a no-op but
>> there are other IP blocks with an prepare_suspend() callback like DCN.
>>
>> That is I think you're destroying the optimization from commit
>> 50e0bae34fa6b ("drm/amd/display: Add and use new dm_prepare_suspend()
>> callback") by adding this code here.
>>
> I guess this takes care of the prepare() before a power_off(). For the
> hibernate prepare() call, in_suspend flag will remain false and it
> should get executed. If the device is runtime-suspended already, then
> the path won't be taken. Assuming that's fine.
>
> Thanks,
> Lijo


That's right. Thank you for the clarification, Lijo.

Rafael reminded me yesterday that there are 2 hibernation mode, controlled by `/sys/power/disk`.
The 2 mode will call different callbacks after saving the image:

shutdown mode, hibernate will go through following steps:
1. amdgpu_pmops_prepare
2. amdgpu_pmops_freeze
3. create image
4. amdgpu_pmops_thaw
5. amdgpu_pmops_complete
6. save image

7. amdgpu_pci_shutdown


platform mode, hibernate will go through following steps:
1. amdgpu_pmops_prepare
2. amdgpu_pmops_freeze
3. create image
4. amdgpu_pmops_thaw
5. amdgpu_pmops_complete
6. save image

7. amdgpu_pmops_prepare
8. amdgpu_pmops_poweroff

This code is just to skip the step 7 of platform mode hibernation, 
prepare to power off.

adev->in_suspend is false in step 2, so it will not return in new the check.
adev->in_suspend is true in step 7, it will return in new the check.

Regards
Sam


>
>>>        /* Return a positive number here so
>>>         * DPM_FLAG_SMART_SUSPEND works properly
>>>         */
>>> @@ -2655,12 +2663,21 @@ static int amdgpu_pmops_thaw(struct device *dev)
>>>    {
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>>    +    /* do not resume device if it's normal hibernation */
>>> +    if (!pm_hibernate_is_recovering())
>>> +        return 0;
>>> +
>>>        return amdgpu_device_resume(drm_dev, true);
>>>    }
>>>      static int amdgpu_pmops_poweroff(struct device *dev)
>>>    {
>>>        struct drm_device *drm_dev = dev_get_drvdata(dev);
>>> +    struct amdgpu_device *adev = drm_to_adev(drm_dev);
>>> +
>>> +    /* device maybe not resumed here, return immediately in this case */
>>> +    if (adev->in_s4 && adev->in_suspend)
>>> +        return 0;
>>>          return amdgpu_device_suspend(drm_dev, true);
>>>    }