linux-kernel - Re: [PATCH] drm/ast: Fix big-endian support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <c5e4fd12-38d9-4169-88d1-115cc38ca409@mailbox.org>
Date: Fri, 12 Dec 2025 16:14:37 +0100
From: Michel Dänzer <michel.daenzer@...lbox.org>
To: Thomas Zimmermann <tzimmermann@...e.de>, René Rebe
 <rene@...ctco.de>
Cc: tpearson@...torengineering.com, dri-devel@...ts.freedesktop.org,
 linux-kernel@...r.kernel.org, airlied@...hat.com
Subject: Re: [PATCH] drm/ast: Fix big-endian support

On 12/11/25 15:56, Thomas Zimmermann wrote:
> Am 11.12.25 um 15:31 schrieb René Rebe:
>> On Thu, 11 Dec 2025 15:03:48 +0100, Thomas Zimmermann <tzimmermann@...e.de> wrote:
>>
>>>       /* write checksum + signature */
>>> +    writel(swab32(csum), dst);
>>> +    writel(swab32(width), dst + AST_HWC_SIGNATURE_SizeX);
>>> +    writel(swab32(height), dst + AST_HWC_SIGNATURE_SizeY);
>>> +    writel(swab32(0), dst + AST_HWC_SIGNATURE_HOTSPOTX);
>>> +    writel(swab32(0), dst + AST_HWC_SIGNATURE_HOTSPOTY);
>>> +#else
>>> +    memcpy_toio(dst, src, AST_HWC_SIZE);
>>>       dst += AST_HWC_SIZE;
>>> +
>>> +    /* write checksum + signature */
>>>       writel(csum, dst);
>>>       writel(width, dst + AST_HWC_SIGNATURE_SizeX);
>>>       writel(height, dst + AST_HWC_SIGNATURE_SizeY);
>>>       writel(0, dst + AST_HWC_SIGNATURE_HOTSPOTX);
>>>       writel(0, dst + AST_HWC_SIGNATURE_HOTSPOTY);
>>> +#endif
>> I'm pretty sure this will break the cursor, as the position was
>> working correctly and I only had to swap the cursor image data. The
>> csum will also not be identical anyway, as the checksum function
>> computes it in native byte order. Theoretically that would have to be
>> changed. However, I do not see where it is really used, maybe only
>> some special remote desktop vendor protocol that I'm not using. Maybe
>> the exact checksum does not even matter and is only used as
>> optimization to not resend an unchanged cursor image.
> 
> Oh well! I though that the bus does implicit byte swaps? Or does writel() already swap to little endian, which the AST chip expects? I'm confused.

FWIW, writel indeed converts from native byte order to little endian, see include/asm-generic/io.h.

Can't help with your other questions though.


-- 
Earthling Michel Dänzer       \        GNOME / Xwayland / Mesa developer
https://redhat.com             \               Libre software enthusiast