linux-kernel - Re: [PATCH 2/2] mm/slub: don't use reserved highatomic pageblock for optimistic try

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 29 Aug 2017 09:33:44 +0900
From:   Joonsoo Kim <iamjoonsoo.kim@....com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Vlastimil Babka <vbabka@...e.cz>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Christoph Lameter <cl@...ux.com>,
        Pekka Enberg <penberg@...nel.org>,
        David Rientjes <rientjes@...gle.com>, linux-mm@...ck.org,
        linux-kernel@...r.kernel.org,
        Mel Gorman <mgorman@...hsingularity.net>
Subject: Re: [PATCH 2/2] mm/slub: don't use reserved highatomic pageblock for
 optimistic try

On Mon, Aug 28, 2017 at 03:08:29PM +0200, Michal Hocko wrote:
> On Mon 28-08-17 13:29:29, Vlastimil Babka wrote:
> > On 08/28/2017 03:11 AM, js1304@...il.com wrote:
> > > From: Joonsoo Kim <iamjoonsoo.kim@....com>
> > > 
> > > High-order atomic allocation is difficult to succeed since we cannot
> > > reclaim anything in this context. So, we reserves the pageblock for
> > > this kind of request.
> > > 
> > > In slub, we try to allocate higher-order page more than it actually
> > > needs in order to get the best performance. If this optimistic try is
> > > used with GFP_ATOMIC, alloc_flags will be set as ALLOC_HARDER and
> > > the pageblock reserved for high-order atomic allocation would be used.
> > > Moreover, this request would reserve the MIGRATE_HIGHATOMIC pageblock
> > > ,if succeed, to prepare further request. It would not be good to use
> > > MIGRATE_HIGHATOMIC pageblock in terms of fragmentation management
> > > since it unconditionally set a migratetype to request's migratetype
> > > when unreserving the pageblock without considering the migratetype of
> > > used pages in the pageblock.
> > > 
> > > This is not what we don't intend so fix it by unconditionally setting
> > > __GFP_NOMEMALLOC in order to not set ALLOC_HARDER.
> > 
> > I wonder if it would be more robust to strip GFP_ATOMIC from alloc_gfp.
> > E.g. __GFP_NOMEMALLOC does seem to prevent ALLOC_HARDER, but not
> > ALLOC_HIGH. Or maybe we should adjust __GFP_NOMEMALLOC implementation
> > and document it more thoroughly? CC Michal Hocko
> 
> Yeah, __GFP_NOMEMALLOC is rather inconsistent. It has been added to
> override __GFP_MEMALLOC resp. PF_MEMALLOC AFAIK. In this particular
> case I would agree that dropping __GFP_HIGH and __GFP_ATOMIC would
> be more precise. I am not sure we want to touch the existing semantic of
> __GFP_NOMEMALLOC though. This would require auditing all the existing
> users (something tells me that quite some of those will be incorrect...)

Hmm... now I realize that there is another reason that we need to use
__GFP_NOMEMALLOC. Even if this allocation comes from PF_MEMALLOC user,
this optimistic try should not use the reserved memory below the
watermark. That is, it should not use ALLOC_NO_WATERMARKS. It can
only be accomplished by using __GFP_NOMEMALLOC.

> 
> > Also, were these 2 patches done via code inspection or you noticed
> > suboptimal behavior which got fixed? Thanks.
> 
> The patch description is not very clear to me either but I guess that
> Joonsoo sees to many larger order pages to back slab objects when the
> system is not under heavy memory pressure and that increases internal
> fragmentation?

Your guess is right. I found this problem when I checked the
fragmentation ratio through the benchmark some months ago. I don't
remember detailed system state in that benchmark.

Thanks.