linux-kernel - Re: 6.11/regression/bisected - after commit 1b04dcca4fb1, launching some RenPy games causes computer hang

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6db472e4-cd90-4ba6-8368-725b10ba5b4a@amd.com>
Date: Tue, 10 Sep 2024 17:11:42 -0400
From: Leo Li <sunpeng.li@....com>
To: Mikhail Gavrilov <mikhail.v.gavrilov@...il.com>
Cc: Harry Wentland <harry.wentland@....com>, zaeem.mohamed@....com,
 pekka.paalanen@...labora.com, "Wheeler, Daniel" <daniel.wheeler@....com>,
 "Deucher, Alexander" <alexander.deucher@....com>,
 amd-gfx list <amd-gfx@...ts.freedesktop.org>,
 dri-devel <dri-devel@...ts.freedesktop.org>,
 Linux List Kernel Mailing <linux-kernel@...r.kernel.org>,
 Linux regressions mailing list <regressions@...ts.linux.dev>
Subject: Re: 6.11/regression/bisected - after commit 1b04dcca4fb1, launching
 some RenPy games causes computer hang

Hi Mikhail,

Can you give this patch a try to see if it helps?
https://gist.github.com/leeonadoh/3271e90ec95d768424c572c970ada743

Thanks,
Leo

On 2024-09-10 11:47, Leo Li wrote:
> 
> 
> On 2024-09-08 19:30, Mikhail Gavrilov wrote:
>> I have done additional tests:
>> 1. The computer does not hang with 6900XT instead the screen flickers
>> when moving the cursor.
>> 2. The computer does not hang with 7900XTX if I turn off VRR. But the
>> screen flickers when moving the cursor, as on 6900XT.
>> To enable VRR, please set 'variable-refresh-rate' in
>> experimental-features, and in the Display setting, enable Variable
>> Refresh Rate.
>> $ gsettings set org.gnome.mutter experimental-features
>> "['variable-refresh-rate', 'scale-monitor-framebuffer']"
>> https://postimg.cc/PvXYdvGR
> 
> Thanks Mikhail, I think I know what's going on now.
> 
> The `scale-monitor-framebuffer` experimental setting is what puts us down the
> bad code path. It seems VRR has nothing to do with this issue, just setting
> `scale-monitor-framebuffer` is enough to reproduce.
> 
> It seems that mutter with this setting is opting for HW scaling rather than GPU
> scaling. I see that "Find the Orange Narwhal" sends out a 1080p buffer,
> which with this setting, gets directly scanned out and scaled by DCN HW to 4k in
> full screen.
> 
> An oddity with current gen DCN hardware is that the cursor inherits the scaling
> of the HW plane underneath. So if mutter requests a hw cursor with a different
> scaling than the game's plane, amdgpu will reject that, and likely force mutter
> into SW cursor.
> 
> My offending patch changed this behavior by rerouting DCN HW pipes to
> accommodate such a configuration. It essentially takes a full-fledged DCN
> overlay plane, and uses that just for the cursor, and thereby freeing it from
> inheriting things from the underlying hw plane.
> 
> My guess is this causes flickering due to how DC (display core driver) handles
> updates; it needs all enabled planes in it's update state. However, a KMS cursor
> update will only include the cursor plane. It's likely that amdgpu_dm only adds
> the dedicated cursor plane to DC's update state, leaving the game's plane out.
> 
> The fix isn't exactly trivial. If I don't get anywhere before the fixes window,
> I'll send out a revert.
> 
> Cheers,
> Leo