linux-kernel - [PATCH] The patch solves the type error of the parameter “off” in syscall mmap on the ARM64 platform.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1555504525-17962-1-git-send-email-zhouby_cn@126.com>
Date:   Wed, 17 Apr 2019 20:35:25 +0800
From:   Boyang Zhou <zhouby_cn@....com>
To:     catalin.marinas@....com
Cc:     will.deacon@....com, haozhu@...t.edu.cn, mark.rutland@....com,
        linux@...inikbrodowski.net, linux-arm-kernel@...ts.infradead.org,
        linux-kernel@...r.kernel.org, Boyang Zhou <zhouby_cn@....com>
Subject: [PATCH] The patch solves the type error  of the parameter “off” in syscall mmap on the ARM64 platform.

The error information is that “offset value too large for defined data type”.
Reason:
On the X86 platform, the data type of “off" is unsigned long; but on the ARM64 platform, the data type is defined as off_t, and off_t is by type long instead of unsigned long.
When the off right shifts in the function “sys_mmap_pgoff(addr, len, prot, flags, fd, off >> PAGE_SHIFT)"on ARM64, high address of off is filled with sign bit 1instead of 0.
In our case, we mmap GPU doorbell on both platform. On the x86 platform, the value of off is f009c00000000000, after shift the value becomes f009c00000000; while on the ARM64, the value of off changes from ed35c00000000000 to fffed35c00000000. This value is treated as unsigned long in later functions. So it is too big for off and the error happened.
We have tested the patchs in Huawei ARM64 server with a couples of AMD GPUs.

Signed-off-by: Boyang Zhou <zhouby_cn@....com>
---
 arch/arm64/kernel/sys.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/arm64/kernel/sys.c b/arch/arm64/kernel/sys.c
index b44065f..6f91e81 100644
--- a/arch/arm64/kernel/sys.c
+++ b/arch/arm64/kernel/sys.c
@@ -31,7 +31,7 @@

 SYSCALL_DEFINE6(mmap, unsigned long, addr, unsigned long, len,
 		unsigned long, prot, unsigned long, flags,
-		unsigned long, fd, off_t, off)
+		unsigned long, fd, unsigned long, off)
 {
 	if (offset_in_page(off) != 0)
 		return -EINVAL;
-- 
2.7.4