linux-kernel - Re: [PATCH] hugetlbfs: Fix integer overflow check in hugetlbfs_file

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 13 Jul 2023 15:55:55 +0800
From:   linke li <lilinke99@...il.com>
To:     David Hildenbrand <david@...hat.com>
Cc:     Linke Li <lilinke99@...mail.com>, linux-mm@...ck.org,
        llvm@...ts.linux.dev, linux-kernel@...r.kernel.org,
        trix@...hat.com, ndesaulniers@...gle.com, nathan@...nel.org,
        muchun.song@...ux.dev, mike.kravetz@...cle.com
Subject: Re: [PATCH] hugetlbfs: Fix integer overflow check in hugetlbfs_file_mmap()

> So we're adding code to handle eventual future compiler bugs? That sounds
> wrong, but maybe I misunderstood the problem you are trying to solve?

Sorry for not making it clear. My focus is the presence of undefined
behavior in kernel code.
Compilers can generate any code for undefined behavior and compiler
developers will not
take this as compiler bugs. In my option,  kernel should not have
undefined behavior.

I double check this patch, this patch can not solve this issue well. I
am considering a new
patch below. The new patch do overflow check before the addition operation.
```
--- a/fs/hugetlbfs/inode.c
+++ b/fs/hugetlbfs/inode.c
@@ -155,10 +155,10 @@ static int hugetlbfs_file_mmap(struct file
*file, struct vm_area_struct *vma)
  return -EINVAL;

  vma_len = (loff_t)(vma->vm_end - vma->vm_start);
- len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
  /* check for overflow */
- if (len < vma_len)
+ if (vma_len > LLONG_MAX - ((loff_t)vma->vm_pgoff << PAGE_SHIFT))
  return -EINVAL;
+ len = vma_len + ((loff_t)vma->vm_pgoff << PAGE_SHIFT);
```