linux-kernel - Re: [PATCH] drm/ast: Fix big-endian support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20251210.175635.1285090497167018958.rene@exactco.de>
Date: Wed, 10 Dec 2025 17:56:35 +0100 (CET)
From: René Rebe <rene@...ctco.de>
To: tzimmermann@...e.de
Cc: tpearson@...torengineering.com, dri-devel@...ts.freedesktop.org,
 linux-kernel@...r.kernel.org, airlied@...hat.com
Subject: Re: [PATCH] drm/ast: Fix big-endian support

Hi,

On Wed, 10 Dec 2025 16:41:50 +0100, Thomas Zimmermann <tzimmermann@...e.de> wrote:

> Hi
> 
> Am 10.12.25 um 16:33 schrieb René Rebe:
> > Hallo,
> >
> > On Wed, 10 Dec 2025 09:55:50 +0100, Thomas Zimmermann
> > <tzimmermann@...e.de> wrote:
> > ...
> >>>> Does that fix the color corruption?
> >>> Following your suggestions conversion does not want to just work:
> >>>
> >>> root@...DE_SPARC_T4_1:~# dmesg  | tail
> >>> [  105.444761] ast 0000:0a:00.0: AST 2200 detected
> >>> [ 105.444947] ast 0000:0a:00.0: [drm] dram MCLK=266 Mhz type=2
> >>> bus_width=32
> >>> [  105.444963] ast 0000:0a:00.0: [drm] Using analog VGA
> >>> [  105.445470] [drm] Initialized ast 0.1.0 for 0000:0a:00.0 on minor 0
> >>> [ 105.673289] ast 0000:0a:00.0: [drm] format BX24 little-endian
> >>> (0x34325842) not supported
> >>> [  105.673302] ast 0000:0a:00.0: [drm] No compatible format found
> >>> [ 105.673348] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
> >>> emulation (ret=-22)
> >>> [ 105.901306] ast 0000:0a:00.0: [drm] format BX24 little-endian
> >>> (0x34325842) not supported
> >>> [  105.901319] ast 0000:0a:00.0: [drm] No compatible format found
> >>> [ 105.901350] ast 0000:0a:00.0: [drm] *ERROR* fbdev: Failed to setup
> >>> emulation (ret=-22)
> >> Oh well...
> >>
> >> There's a very simple patch attach. Does it fix the problem?
> > Yes, only leaving the hardcoded swapping from my patch liek this fixes
> > the byte-swapped output as expected on the sparc64 Sun T4.
> 
> Great.
> 
> >
> > How would you like me to go from here? Just use the chip_id to force
> > swapping and enable hw swapper for pre-AST2400 chips or fix the
> > generic format selection to work as you had suggested?
> >
> > Does the ast_primary_plane_formats need to byte swapped formats for it
> > to work?
> 
> I'll send out a full patch that implements the byte swapping. Once
> reviewed, it can be merged quickly. Can I add your Tested-by tag to
> the patch?

I'd be happy to finish my work. But if you want to put the last touch
on it now you can add Co-developed-by, too ... and I'll test the final
version.

   René

-- 
René Rebe, ExactCODE GmbH, Berlin, Germany
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe