linux-kernel - [PATCH] hugetlbfs: Fix integer overflow check in hugetlbfs_file

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 10 Jul 2023 16:32:40 +0800
From:   Linke Li <lilinke99@...mail.com>
To:     linux-mm@...ck.org
Cc:     llvm@...ts.linux.dev, linux-kernel@...r.kernel.org,
        trix@...hat.com, ndesaulniers@...gle.com, nathan@...nel.org,
        muchun.song@...ux.dev, mike.kravetz@...cle.com,
        Linke Li <lilinke99@...il.com>
Subject: [PATCH] hugetlbfs: Fix integer overflow check in  hugetlbfs_file_mmap()

From: Linke Li <lilinke99@...il.com>

	vma_len = (loff_t)(vma->vm_end - vma->vm_start);
	len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
	/* check for overflow */
	if (len < vma_len)
		return -EINVAL;

The existing code includes an integer overflow check, which indicates
that the variable len has the potential to overflow, leading to undefined
 behavior according to the C standard. However, both GCC and Clang
compilers may eliminate this overflow check based on the assumption
that there will be no undefined behavior. Although the Linux kernel
disables these optimizations by using the -fno-strict-overflow option,
there is still a risk if the compilers make mistakes in the future.

To address this potential issue, this patch introduces a new check
that effectively prevents integer overflow. This check ensures the
safe operation of the code even when the Linux kernel is compiled
without the -fno-strict-overflow option, providing an added layer
of protection against potential issues caused by compiler behavior.

Signed-off-by: Linke Li <lilinke99@...il.com>
---
 fs/hugetlbfs/inode.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c
index 7b17ccfa039d..1b4648a8e296 100644
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -157,7 +157,7 @@ static int hugetlbfs_file_mmap(struct file *file, struct vm_area_struct *vma)
 	vma_len = (loff_t)(vma->vm_end - vma->vm_start);
 	len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
 	/* check for overflow */
-	if (len < vma_len)
+	if (vma_len > LLONG_MAX - ((loff_t)vma->vm_pgoff << PAGE_SHIFT))
 		return -EINVAL;

 	inode_lock(inode);
-- 
2.25.1