linux-kernel - Re: Possible 6.5 regression: Huge values for "commited memory"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <30f16b4f-a2fa-fc42-fe6e-abad01c3f794@virtuozzo.com>
Date:   Fri, 15 Sep 2023 19:27:58 +0300
From:   Michael Labiuk <michael.labiuk@...tuozzo.com>
To:     Bagas Sanjaya <bagasdotme@...il.com>,
        Christoph Biedl <linux-kernel.bfrz@...chmal.in-ulm.de>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Linux PARISC <linux-parisc@...r.kernel.org>,
        Linux Memory Management List <linux-mm@...ck.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Linux Regressions <regressions@...ts.linux.dev>,
        "Liam R. Howlett" <Liam.Howlett@...cle.com>,
        "Matthew Wilcox (Oracle)" <willy@...radead.org>
Subject: Re: Possible 6.5 regression: Huge values for "commited memory"

I am also reproducing this issue.

Memory overcommit is disabled in my setup. Committed_AS grows to 
mem+swap after 1-2 hours of compilation load and system becomes 
nonoperational.

I've bisected this issue:

$ git bisect log
git bisect start
# status: waiting for both good and bad commits
# bad: [2dde18cd1d8fac735875f2e4987f11817cc0bc2c] Linux 6.5
git bisect bad 2dde18cd1d8fac735875f2e4987f11817cc0bc2c
# status: waiting for good commit(s), bad commit known
# good: [f60d5fd5e950c89a38578ae6f25877de511bb031] Linux 6.4.15
git bisect good f60d5fd5e950c89a38578ae6f25877de511bb031
# good: [6995e2de6891c724bfeb2db33d7b87775f913ad1] Linux 6.4
git bisect good 6995e2de6891c724bfeb2db33d7b87775f913ad1
# good: [b775d6c5859affe00527cbe74263de05cfe6b9f9] Merge tag 'mips_6.5' 
of git://git.kernel.org/pub/scm/linux/kernel/git/mips/linux
git bisect good b775d6c5859affe00527cbe74263de05cfe6b9f9
# bad: [56cbceab928d7ac3702de172ff8dcc1da2a6aaeb] Merge tag 
'usb-6.5-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/usb
git bisect bad 56cbceab928d7ac3702de172ff8dcc1da2a6aaeb
# good: [b30d7a77c53ec04a6d94683d7680ec406b7f3ac8] Merge tag 
'perf-tools-for-v6.5-1-2023-06-28' of 
git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next
git bisect good b30d7a77c53ec04a6d94683d7680ec406b7f3ac8
# bad: [dfab92f27c600fea3cadc6e2cb39f092024e1fef] Merge tag 
'nfs-for-6.5-1' of git://git.linux-nfs.org/projects/trondmy/linux-nfs
git bisect bad dfab92f27c600fea3cadc6e2cb39f092024e1fef
# good: [28968f384be3c064d66954aac4c534a5e76bf973] Merge tag 
'pinctrl-v6.5-1' of 
git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl
git bisect good 28968f384be3c064d66954aac4c534a5e76bf973
# good: [5d95ff84e62be914b4a4dabfa814e4096b05b1b0] Merge tag 'v6.5-p1' 
of git://git.kernel.org/pub/scm/linux/kernel/git/herbert/crypto-2.6
git bisect good 5d95ff84e62be914b4a4dabfa814e4096b05b1b0
# bad: [d25f002575146d67b5ebea541e6db3696c957c25] Merge tag 
'cxl-for-6.5' of git://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl
git bisect bad d25f002575146d67b5ebea541e6db3696c957c25
# bad: [0a1c979c6b7dfe5b6c105d0f0f9f068b5eb07e25] Merge tag 
'libnvdimm-for-6.5' of 
git://git.kernel.org/pub/scm/linux/kernel/git/nvdimm/nvdimm
git bisect bad 0a1c979c6b7dfe5b6c105d0f0f9f068b5eb07e25
# good: [a507db1d8fdc39802415e4d2ef6d1aecd67927fa] Merge tag 
'6.5-rc-smb3-client-fixes-part1' of git://git.samba.org/sfrench/cifs-2.6
git bisect good a507db1d8fdc39802415e4d2ef6d1aecd67927fa
# good: [46e66dab8565f742374e9cc4ff7d35f344d774e2] dax/kmem: Pass valid 
argument to memory_group_register_static
git bisect good 46e66dab8565f742374e9cc4ff7d35f344d774e2
# bad: [170ab6c51a42a8a1c1a7ce09367b578db6f2f383] Merge tag 
'flex-array-transformations-6.5-rc1' of 
git://git.kernel.org/pub/scm/linux/kernel/git/gustavoars/linux
git bisect bad 170ab6c51a42a8a1c1a7ce09367b578db6f2f383
# bad: [408579cd627a15bd703fe3eeb8485fd02726e9d3] mm: Update 
do_vmi_align_munmap() return semantics
git bisect bad 408579cd627a15bd703fe3eeb8485fd02726e9d3
# good: [e4bd84c069f212c01258e405f86e91f327888e41] mm: Always downgrade 
mmap_lock if requested
git bisect good e4bd84c069f212c01258e405f86e91f327888e41
# first bad commit: [408579cd627a15bd703fe3eeb8485fd02726e9d3] mm: 
Update do_vmi_align_munmap() return semantics


commit 408579cd627a15bd703fe3eeb8485fd02726e9d3
Author: Liam R. Howlett
Date:   Thu Jun 29 22:28:16 2023 -0400

     mm: Update do_vmi_align_munmap() return semantics

     Since do_vmi_align_munmap() will always honor the downgrade request on
     the success, the callers no longer have to deal with confusing return
     codes.  Since all callers that request downgrade actually want the lock
     to be dropped, change the downgrade to an unlock request.

     Note that the lock still needs to be held in read mode during the page
     table clean up to avoid races with a map request.

     Update do_vmi_align_munmap() to return 0 for success.  Clean up the
     callers and comments to always expect the unlock to be honored on the
     success path.  The error path will always leave the lock untouched.

     As part of the cleanup, the wrapper function do_vmi_munmap() and 
callers
     to the wrapper are also updated.

Looks like memory statistics depends on removed locks.

Thanks.