linux-kernel - [PATCH] drm/ast: Fix big-endian support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <20251202.170626.2134482663677806825.rene@exactco.de>
Date: Tue, 02 Dec 2025 17:06:26 +0100 (CET)
From: René Rebe <rene@...ctco.de>
To: dri-devel@...ts.freedesktop.org, linux-kernel@...r.kernel.org
Cc: Dave Airlie <airlied@...hat.com>, Thomas Zimmermann
 <tzimmermann@...e.de>
Subject: [PATCH] drm/ast: Fix big-endian support

The Aspeed ast drm driver has the frame-buffer RGBX swapped on
big-endian RISC systems. Fix by enabling byte swapping for any
__BIG_ENDIAN config.

Fixes: 12fec1405dd5 ("drm: Initial KMS driver for AST (ASpeed Technologies) 2000 series (v2)")
Signed-off-by: René Rebe <rene@...ctco.de>
---
Tested on Oracle T4-1 running sparc64 T2/Linux.
---
 drivers/gpu/drm/ast/ast_mode.c | 14 ++++++++++++++
 drivers/gpu/drm/ast/ast_reg.h  |  6 ++++++
 2 files changed, 20 insertions(+)

diff --git a/drivers/gpu/drm/ast/ast_mode.c b/drivers/gpu/drm/ast/ast_mode.c
index 30b011ed0a05..155ae35470d9 100644
--- a/drivers/gpu/drm/ast/ast_mode.c
+++ b/drivers/gpu/drm/ast/ast_mode.c
@@ -708,6 +708,20 @@ static void ast_crtc_helper_mode_set_nofb(struct drm_crtc *crtc)
 	ast_set_dclk_reg(ast, adjusted_mode, vmode);
 	ast_set_crtthd_reg(ast);
 	ast_set_sync_reg(ast, adjusted_mode, vmode);
+
+#ifdef __BIG_ENDIAN
+	/* Big-endian byte-swapping */
+	switch (ast_crtc_state->format->format) {
+	case DRM_FORMAT_RGB565:
+		ast_set_index_reg_mask(ast, AST_IO_VGACRI, AST_IO_VGACRA2, 0x3f, 0x40);
+		break;
+	case DRM_FORMAT_XRGB8888:
+		ast_set_index_reg_mask(ast, AST_IO_VGACRI, AST_IO_VGACRA2, 0x3f, 0x80);
+		break;
+	default:
+		break;
+	}
+#endif
 }
 
 static int ast_crtc_helper_atomic_check(struct drm_crtc *crtc,
diff --git a/drivers/gpu/drm/ast/ast_reg.h b/drivers/gpu/drm/ast/ast_reg.h
index 30578e3b07e4..5c8c0fd2e229 100644
--- a/drivers/gpu/drm/ast/ast_reg.h
+++ b/drivers/gpu/drm/ast/ast_reg.h
@@ -75,4 +75,10 @@
 #define AST_IO_VGAIR1_R			(0x5A)
 #define AST_IO_VGAIR1_VREFRESH		BIT(3)
 
+/*
+ * PCI Control
+ */
+
+#define AST_IO_VGACRA2			(0xA2) /* PCI control & big-endian */
+
 #endif
-- 
2.52.0


-- 
René Rebe, ExactCODE GmbH, Berlin, Germany
https://exactco.de • https://t2linux.com • https://patreon.com/renerebe