linux-kernel - Re: [PATCH] drm/ast: Fix big-endian support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <dda9846d-3893-43ab-9cce-12a7f41fb974@suse.de>
Date: Wed, 10 Dec 2025 16:41:50 +0100
From: Thomas Zimmermann <tzimmermann@...e.de>
To: René Rebe <rene@...ctco.de>
Cc: tpearson@...torengineering.com, dri-devel@...ts.freedesktop.org,
 linux-kernel@...r.kernel.org, airlied@...hat.com
Subject: Re: [PATCH] drm/ast: Fix big-endian support

Hi

Am 10.12.25 um 16:33 schrieb René Rebe:
> Hallo,
>
> On Wed, 10 Dec 2025 09:55:50 +0100, Thomas Zimmermann <tzimmermann@...e.de> wrote:
> ...
>>>> Does that fix the color corruption?
>>> Following your suggestions conversion does not want to just work:
>>>
>>> root@...DE_SPARC_T4_1:~# dmesg  | tail
>>> [  105.444761] ast 0000:0a:00.0: AST 2200 detected
>>> [ 105.444947] ast 0000:0a:00.0: [drm] dram MCLK=266 Mhz type=2
>>> bus_width=32
>>> [  105.444963] ast 0000:0a:00.0: [drm] Using analog VGA
>>> [  105.445470] [drm] Initialized ast 0.1.0 for 0000:0a:00.0 on minor 0
>>> [ 105.673289] ast 0000:0a:00.0: [drm] format BX24 little-endian
>>> (0x34325842) not supported
>>> [  105.673302] ast 0000:0a:00.0: [drm] No compatible format found
>>> [ 105.673348] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
>>> emulation (ret=-22)
>>> [ 105.901306] ast 0000:0a:00.0: [drm] format BX24 little-endian
>>> (0x34325842) not supported
>>> [  105.901319] ast 0000:0a:00.0: [drm] No compatible format found
>>> [ 105.901350] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
>>> emulation (ret=-22)
>> Oh well...
>>
>> There's a very simple patch attach. Does it fix the problem?
> Yes, only leaving the hardcoded swapping from my patch liek this fixes
> the byte-swapped output as expected on the sparc64 Sun T4.

Great.

>
> How would you like me to go from here? Just use the chip_id to force
> swapping and enable hw swapper for pre-AST2400 chips or fix the
> generic format selection to work as you had suggested?
>
> Does the ast_primary_plane_formats need to byte swapped formats for it
> to work?

I'll send out a full patch that implements the byte swapping. Once 
reviewed, it can be merged quickly. Can I add your Tested-by tag to the 
patch?

Best regards
Thomas

>
> Thanks for sticking with it!
>
>         René
>
>> Best regards
>> Thomas

-- 
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)