linux-kernel - Re: [PATCH 1/5] mm: vmscan: shrink all slab objects if tight on memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20140113150502.4505f661589a4a2d30e6f11d@linux-foundation.org>
Date:	Mon, 13 Jan 2014 15:05:02 -0800
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Vladimir Davydov <vdavydov@...allels.com>
Cc:	<linux-kernel@...r.kernel.org>, <linux-mm@...ck.org>,
	<devel@...nvz.org>, Mel Gorman <mgorman@...e.de>,
	Michal Hocko <mhocko@...e.cz>,
	Johannes Weiner <hannes@...xchg.org>,
	Rik van Riel <riel@...hat.com>,
	Dave Chinner <dchinner@...hat.com>,
	Glauber Costa <glommer@...il.com>
Subject: Re: [PATCH 1/5] mm: vmscan: shrink all slab objects if tight on
 memory

On Sat, 11 Jan 2014 16:36:31 +0400 Vladimir Davydov <vdavydov@...allels.com> wrote:

> When reclaiming kmem, we currently don't scan slabs that have less than
> batch_size objects (see shrink_slab_node()):
> 
>         while (total_scan >= batch_size) {
>                 shrinkctl->nr_to_scan = batch_size;
>                 shrinker->scan_objects(shrinker, shrinkctl);
>                 total_scan -= batch_size;
>         }
> 
> If there are only a few shrinkers available, such a behavior won't cause
> any problems, because the batch_size is usually small, but if we have a
> lot of slab shrinkers, which is perfectly possible since FS shrinkers
> are now per-superblock, we can end up with hundreds of megabytes of
> practically unreclaimable kmem objects. For instance, mounting a
> thousand of ext2 FS images with a hundred of files in each and iterating
> over all the files using du(1) will result in about 200 Mb of FS caches
> that cannot be dropped even with the aid of the vm.drop_caches sysctl!

True.  I suspect this was an accidental consequence of the chosen
implementation.  As you mentioned, I was thinking that the caches would
all be large, and the remaining 1 ..  SHRINK_BATCH-1 objects just
didn't matter.

> This problem was initially pointed out by Glauber Costa [*]. Glauber
> proposed to fix it by making the shrink_slab() always take at least one
> pass, to put it simply, turning the scan loop above to a do{}while()
> loop. However, this proposal was rejected, because it could result in
> more aggressive and frequent slab shrinking even under low memory
> pressure when total_scan is naturally very small.

Well, it wasn't "rejected" - Mel pointed out that Glauber's change
could potentially trigger problems which already exist in shrinkers.

The potential issues seem pretty unlikely to me, and they're things we
can fix up if they eventuate.

So I'm thinking we should at least try Glauber's approach - it's a bit
weird that we should treat the final 0 ..  batch_size-1 objects in a
different manner from all the others.


That being said, I think I'll schedule this patch as-is for 3.14.  Can
you please take a look at implementing the simpler approach, send me
something for 3.15-rc1?

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/