linux-kernel - Re: [PATCH v3] mm: Proactive compaction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <20200420202615.daa216426294e842cb0b523c@linux-foundation.org>
Date:   Mon, 20 Apr 2020 20:26:15 -0700
From:   Andrew Morton <akpm@...ux-foundation.org>
To:     Nitin Gupta <nigupta@...dia.com>
Cc:     Mel Gorman <mgorman@...hsingularity.net>,
        Michal Hocko <mhocko@...e.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Matthew Wilcox <willy@...radead.org>,
        Mike Kravetz <mike.kravetz@...cle.com>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        David Rientjes <rientjes@...gle.com>,
        Nitin Gupta <nitin@...ingupta.dev>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        linux-mm <linux-mm@...ck.org>,
        Linux API <linux-api@...r.kernel.org>
Subject: Re: [PATCH v3] mm: Proactive compaction

On Tue, 10 Mar 2020 15:25:39 -0700 Nitin Gupta <nigupta@...dia.com> wrote:

> For some applications we need to allocate almost all memory as
> hugepages. However, on a running system, higher order allocations can
> fail if the memory is fragmented. Linux kernel currently does on-demand
> compaction as we request more hugepages but this style of compaction
> incurs very high latency. Experiments with one-time full memory
> compaction (followed by hugepage allocations) shows that kernel is able
> to restore a highly fragmented memory state to a fairly compacted memory
> state within <1 sec for a 32G system. Such data suggests that a more
> proactive compaction can help us allocate a large fraction of memory as
> hugepages keeping allocation latencies low.

hn, there was plenty of feedback for earlier versions, but then
everyone went quiet.  I guess it's time for a refresh and resend,
please.

With some code comments, please!  Is the code really so self-evident
that this:

+/* Compact all zones within a node according to proactiveness */

is the only thing which needs to be said about it?  How is the reader
to know what proactive compaction actually *is*?

What does extfrag_for_order() do and what does its return value mean?

Please document /sys/kernel/mm/compaction/proactiveness in the
appropriate place under Documentation/