linux-kernel - Re: mm: kswapd struggles reclaiming the pages on 64GB server

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK-uSPpViZma5CG4znAhQ0=XPqVaj1PG8RRyxQLfXjKSd-DjDQ@mail.gmail.com>
Date:   Mon, 22 Aug 2016 19:16:44 +0100
From:   Andriy Tkachuk <andriy.tkachuk@...gate.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     linux-kernel@...r.kernel.org, Mel Gorman <mgorman@...e.de>,
        linux-mm@...ck.org, Johannes Weiner <hannes@...xchg.org>
Subject: Re: mm: kswapd struggles reclaiming the pages on 64GB server

Hi Michal.

Thank you for the reply.

It looks like the root cause of the problems we are facing is a bit
different, although the ultimate effect is similar - bad swapping
effectiveness.

As far as I could understand, Johannes tries to fix the balancing
between anon and file lists. But in my case it looks like the anon
pages which are idle for a long time and could be swapped out - they
all are just sitting in active list and don't move to inactive without
a chance to be scanned and eventually swapped out. (See the
/proc/vmstat samples and explanations in my prev. mail. BTW, the
samples interval is 10 secs there, not the 5. My typo.)

It looks like in my case the system load activity enters a steady mode
when all the scanned pages from inactive list become referenced very
soon. So kswapd aggresively scans, but mostly the inactive list where
it can hardly find to reclaim anything. So the inactive list is not
shortened and, as result, is not refilled from the active one. That's
why the anon pages from active list are not even get a chance to be
scanned. Note: the zone's inactive_ratio is more than 10 on 64GB RAM
systems, so the inactive list is much smaller than active in my case.

  Andriy

On Wed, Aug 17, 2016 at 12:43 PM, Michal Hocko <mhocko@...nel.org> wrote:
> [CCing linux-mm and Johannes]
>
>
> I haven't looked at your numbers deeply but this smells like the long
> standing problem/limitation we have. We are trying really hard to not
> swap out and rather reclaim the page cache because the swap refault
> tends to be more disruptive in many case. Not all, though, and trashing
> like behavior you see is cetainly undesirable.
>
> Johannes has been looking into that area recently. Have a look at
> https://urldefense.proofpoint.com/v2/url?u=http-3A__lkml.kernel.org_r_20160606194836.3624-2D1-2Dhannes-40cmpxchg.org&d=DQIBAg&c=IGDlg0lD0b-nebmJJ0Kp8A&r=rP2MQ-RHGa6a64ebEAbeV_m6Ae_GOWHWTIpipamZCdE&m=Mxava1puJmDToyZNc62FshgwDC66k26arjHAM6o54yI&s=wmYJ3WdYDc73B7hO75xxvmIk0hDoTUSjGH-KxSC48SA&e=
>
> --
> Michal Hocko
> SUSE Labs