linux-kernel - Re: [PATCH] mm, meminit: Serially initialise deferred memory if trace_buf

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20171115115559.rjb5hy6d6332jgjj@dhcp22.suse.cz>
Date:   Wed, 15 Nov 2017 12:55:59 +0100
From:   Michal Hocko <mhocko@...nel.org>
To:     Mel Gorman <mgorman@...hsingularity.net>
Cc:     Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org, yasu.isimatu@...il.com,
        koki.sanagi@...fujitsu.com
Subject: Re: [PATCH] mm, meminit: Serially initialise deferred memory if
 trace_buf_size is specified

On Wed 15-11-17 08:55:56, Mel Gorman wrote:
> Yasuaki Ishimatsu reported a premature OOM when trace_buf_size=100m was
> specified on a machine with many CPUs. The kernel tried to allocate 38.4GB
> but only 16GB was available due to deferred memory initialisation.
> 
> The allocation context is within smp_init() so there are no opportunities
> to do the deferred meminit earlier. Furthermore, the partial initialisation
> of memory occurs before the size of the trace buffers is set so there is
> no opportunity to adjust the amount of memory that is pre-initialised. We
> could potentially catch when memory is low during system boot and adjust the
> amount that is initialised serially but it's a little clumsy as it would
> require a check in the failure path of the page allocator.  Given that
> deferred meminit is basically a minor optimisation that only benefits very
> large machines and trace_buf_size is somewhat specialised, it follows that
> the most straight-forward option is to go back to serialised meminit if
> trace_buf_size is specified.

Can we instead do a smaller trace buffer in the early stage and then
allocate the rest after the whole memory is initialized? The early
memory init code is quite complex to make it even more so for something
that looks like a borderline useful usecase. Seriously, who is going
need 100M trace buffer _per cpu_ during early boot?

> Reported-and-tested-by: YASUAKI ISHIMATSU <yasu.isimatu@...il.com>
> Signed-off-by: Mel Gorman <mgorman@...hsingularity.net>
-- 
Michal Hocko
SUSE Labs