linux-kernel - Re: Bisected: [PATCH v7 2/8] hugetlb: restructure pool allocations

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20231018224422.GC5181@monkey>
Date:   Wed, 18 Oct 2023 15:44:22 -0700
From:   Mike Kravetz <mike.kravetz@...cle.com>
To:     Sergey Senozhatsky <senozhatsky@...omium.org>
Cc:     linux-mm@...ck.org, linux-kernel@...r.kernel.org,
        Muchun Song <songmuchun@...edance.com>,
        Joao Martins <joao.m.martins@...cle.com>,
        Konrad Dybcio <konradybcio@...nel.org>,
        Oscar Salvador <osalvador@...e.de>,
        David Hildenbrand <david@...hat.com>,
        Miaohe Lin <linmiaohe@...wei.com>,
        David Rientjes <rientjes@...gle.com>,
        Anshuman Khandual <anshuman.khandual@....com>,
        Naoya Horiguchi <naoya.horiguchi@...ux.dev>,
        Barry Song <21cnbao@...il.com>, Michal Hocko <mhocko@...e.com>,
        Matthew Wilcox <willy@...radead.org>,
        Xiongchun Duan <duanxiongchun@...edance.com>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: Bisected: [PATCH v7 2/8] hugetlb: restructure pool allocations

On 10/11/23 15:47, Sergey Senozhatsky wrote:
> On (23/10/05 20:20), Mike Kravetz wrote:
> > Allocation of a hugetlb page for the hugetlb pool is done by the routine
> > alloc_pool_huge_page.  This routine will allocate contiguous pages from
> > a low level allocator, prep the pages for usage as a hugetlb page and
> > then add the resulting hugetlb page to the pool.
> > 
> > In the 'prep' stage, optional vmemmap optimization is done.  For
> > performance reasons we want to perform vmemmap optimization on multiple
> > hugetlb pages at once.  To do this, restructure the hugetlb pool
> > allocation code such that vmemmap optimization can be isolated and later
> > batched.
> > 
> > The code to allocate hugetlb pages from bootmem was also modified to
> > allow batching.
> > 
> > No functional changes, only code restructure.
> 
> I'm afraid there are some functional changes.
> 

Hi Sergey,

Sorry for the delay.  Not ignoring your report but chasing this in
another thread.
https://lore.kernel.org/linux-mm/20231018222003.GA21776@monkey/

Quick question.
Are you using LLVM/clang in your builds?

My guess is that you are hitting the same issue.  That BUG at mm/hugetlb.c:3180
should not be checked/executed unless you allocate gigantic hugetlb pages on
the kernel command line.  Suspect you are not doing this, and loop code is
being run when it should not.
-- 
Mike Kravetz

> [...]
> # good: [9e6c86c616f7d4b166c12f1ea9b69831f85c4a86] hugetlb: optimize update_and_free_pages_bulk to avoid lock cycles
> git bisect good 9e6c86c616f7d4b166c12f1ea9b69831f85c4a86
> # bad: [1d50db09471e2a67272cee6e004ffed380ac571b] Merge branch 'master' of git://linuxtv.org/mchehab/media-next.git
> git bisect bad 1d50db09471e2a67272cee6e004ffed380ac571b
> # good: [80b14e865ea20ddc20aae61e2c106ebb03257cd3] bcachefs: Lower BCH_NAME_MAX to 512
> git bisect good 80b14e865ea20ddc20aae61e2c106ebb03257cd3
> # bad: [31d068de8a0de2c44168bbd8a62da21a7bc76051] Merge branch 'for-linux-next' of git://git.kernel.org/pub/scm/linux/kernel/git/sudeep.holla/linux.git
> git bisect bad 31d068de8a0de2c44168bbd8a62da21a7bc76051
> # bad: [0bb194b29fa6296a74b989e0f7f2db4fc11d8012] Merge branch 'perf-tools-next' of git://git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git
> git bisect bad 0bb194b29fa6296a74b989e0f7f2db4fc11d8012
> # good: [62001c9bf6aad59dc800c16613e5440b9226c252] Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/wpan/wpan.git
> git bisect good 62001c9bf6aad59dc800c16613e5440b9226c252
> # good: [21d856352ab78daffb9d05296b87b570f3afac33] Merge branch 'mm-stable' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
> git bisect good 21d856352ab78daffb9d05296b87b570f3afac33
> # good: [7fded33c6971b6c8e87cbbf48e74536aacca2991] perf test: Add pmu-event test for "Compat" and new event_field.
> git bisect good 7fded33c6971b6c8e87cbbf48e74536aacca2991
> # good: [4c37b665186a5e2907bf0308474ac3f15eb847da] compiler.h: move __is_constexpr() to compiler.h
> git bisect good 4c37b665186a5e2907bf0308474ac3f15eb847da
> # bad: [3424f8f382bd4d30e01eaf316823321ef7bd1560] mm: delete checks for xor_unlock_is_negative_byte()
> git bisect bad 3424f8f382bd4d30e01eaf316823321ef7bd1560
> # bad: [b5c6a60fe5b0339ba9c54f9f871db5c4a0e47906] iomap: protect read_bytes_pending with the state_lock
> git bisect bad b5c6a60fe5b0339ba9c54f9f871db5c4a0e47906
> # bad: [bfae92330ddc44968c628c0085082d25061495a4] hugetlb: batch PMD split for bulk vmemmap dedup
> git bisect bad bfae92330ddc44968c628c0085082d25061495a4
> # bad: [fb59f2cb8956f3888d2e0b438cc503565aa3c405] hugetlb: perform vmemmap optimization on a list of pages
> git bisect bad fb59f2cb8956f3888d2e0b438cc503565aa3c405
> # bad: [bfb41d6b2fe148c939366957ea9cb9aa72d59c4c] hugetlb: restructure pool allocations
> git bisect bad bfb41d6b2fe148c939366957ea9cb9aa72d59c4c
> # first bad commit: [bfb41d6b2fe148c939366957ea9cb9aa72d59c4c] hugetlb: restructure pool allocations
> 
> bfb41d6b2fe148c939366957ea9cb9aa72d59c4c is the first bad commit
> 
> commit bfb41d6b2fe148c939366957ea9cb9aa72d59c4c
> Author: Mike Kravetz
> Date:   Thu Oct 5 20:20:04 2023 -0700
> 
>     hugetlb: restructure pool allocations
> 
> 
> Bugs-on linux-next:
> 
> [    0.827640] ------------[ cut here ]------------
> [    0.828608] kernel BUG at mm/hugetlb.c:3180!