lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 9 Jan 2019 15:11:13 +0100
From:   Michal Hocko <mhocko@...nel.org>
To:     Kirill Tkhai <ktkhai@...tuozzo.com>
Cc:     akpm@...ux-foundation.org, hannes@...xchg.org,
        josef@...icpanda.com, jack@...e.cz, hughd@...gle.com,
        darrick.wong@...cle.com, aryabinin@...tuozzo.com, guro@...com,
        mgorman@...hsingularity.net, shakeelb@...gle.com,
        linux-mm@...ck.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH RFC 0/3] mm: Reduce IO by improving algorithm of memcg
 pagecache pages eviction

On Wed 09-01-19 15:20:18, Kirill Tkhai wrote:
> On nodes without memory overcommit, it's common a situation,
> when memcg exceeds its limit and pages from pagecache are
> shrinked on reclaim, while node has a lot of free memory.

Yes, that is the semantic of the hard limit. If the system is not
overcommitted then the hard limit can be used to prevent unexpected
direct reclaim from unrelated activity.

> Further access to the pages requires real device IO, while
> IO causes time delays, worse powerusage, worse throughput
> for other users of the device, etc.

It is to be expected that a memory throttled usage will have this side
effect IMO.

> Cleancache is not a good solution for this problem, since
> it implies copying of page on every cleancache_put_page()
> and cleancache_get_page(). Also, it requires introduction
> of internal per-cleancache_ops data structures to manage
> cached pages and their inodes relationships, which again
> introduces overhead.
> 
> This patchset introduces another solution. It introduces
> a new scheme for evicting memcg pages:
> 
>   1)__remove_mapping() uncharges unmapped page memcg
>     and leaves page in pagecache on memcg reclaim;
> 
>   2)putback_lru_page() places page into root_mem_cgroup
>     list, since its memcg is NULL. Page may be evicted
>     on global reclaim (and this will be easily, as
>     page is not mapped, so shrinker will shrink it
>     with 100% probability of success);
> 
>   3)pagecache_get_page() charges page into memcg of
>     a task, which takes it first.

But this also means that any hard limited memcg can fill up all the
memory and break the above assumption about the isolation from direct
reclaim. Not to mention the OOM or is there anything you do anything
about preventing that?

That beig said, I do not think we want to or even can change the
semantic of the hard limit and break existing setups. I am still
interested to hear more about more detailed/specific usecases that might
benefit from this behavior. Why do those users even use hard limit at
all? To protect from anon memory leaks? Do different memcgs share the
page cache heavily?
-- 
Michal Hocko
SUSE Labs

Powered by blists - more mailing lists