linux-kernel - Re: [PATCH] drm/ast: Fix big-endian support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20251205.161459.1654100040521559754.rene@exactco.de>
Date: Fri, 05 Dec 2025 16:14:59 +0100 (CET)
From: René Rebe <rene@...ctco.de>
To: tzimmermann@...e.de
Cc: dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org,
 airlied@...hat.com, Timothy Pearson <tpearson@...torengineering.com>
Subject: Re: [PATCH] drm/ast: Fix big-endian support

Hello Thomas,

On Wed, 3 Dec 2025 10:40:17 +0100, Thomas Zimmermann <tzimmermann@...e.de> wrote:

> [2]
> https://gitlab.freedesktop.org/drm/misc/kernel/-/blob/drm-misc-next-2025-12-01-1/drivers/gpu/drm/ast/ast_mode.c?ref_type=tags#L559
> [3]
> https://gitlab.freedesktop.org/drm/misc/kernel/-/blob/drm-misc-next-2025-12-01-1/drivers/gpu/drm/ast/ast_cursor.c?ref_type=tags#L209
> 
> > +	case DRM_FORMAT_RGB565:
> > + ast_set_index_reg_mask(ast, AST_IO_VGACRI, AST_IO_VGACRA2, 0x3f,
> > 0x40);
> > +		break;
> > +	case DRM_FORMAT_XRGB8888

While working on it I discovered that the Big-Endian byte-swapping
bits do apparently not just-work on a newer AST2400 in our Power 8
while my initial patch did work as tested with an AST2200 in the Sun
T4-1 :-/

Maybe that is what Timothy meant with "This is due to a ppc64 hardware
quirk, which when combined with a hardware design fault in the AST2500
VGA controller results in a need to use software-based red-blue
channel swapping." [1]

Is there a way to simply specify the frame-buffer as BGRX8888? In a
quick test the drm layer complaint about "not supported" and "no
compatible format found"?

Thanks,
	René

[1] https://lore.kernel.org/dri-devel/407388289.1798972.1760725035958.JavaMail.zimbra@raptorengineeringinc.com/

-- 
René Rebe, ExactCODE GmbH, Berlin, Germany
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe