linux-kernel - Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <017a11b1-4115-bdf6-6ebe-e121dd03b386@oracle.com>
Date:   Tue, 3 Sep 2019 13:45:21 -0600
From:   Khalid Aziz <khalid.aziz@...cle.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     Bharath Vedartham <linux.bhar@...il.com>,
        akpm@...ux-foundation.org, vbabka@...e.cz,
        mgorman@...hsingularity.net, dan.j.williams@...el.com,
        osalvador@...e.de, richard.weiyang@...il.com, hannes@...xchg.org,
        arunks@...eaurora.org, rppt@...ux.vnet.ibm.com, jgg@...pe.ca,
        amir73il@...il.com, alexander.h.duyck@...ux.intel.com,
        linux-mm@...ck.org, linux-kernel-mentees@...ts.linuxfoundation.org,
        linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

On 9/2/19 2:02 AM, Michal Hocko wrote:
> On Fri 30-08-19 15:35:06, Khalid Aziz wrote:
> [...]
>> - Kernel is not self-tuning and is dependent upon a userspace tool to
>> perform well in a fundamental area of memory management.
> 
> You keep bringing this up without an actual analysis of a wider range of
> workloads that would prove that the default behavior is really
> suboptimal. You are making some assumptions based on a very specific DB
> workload which might benefit from a more aggressive background workload.
> If you really want to sell any changes to auto tuning then you really
> need to come up with more workloads and an actual theory why an early
> and more aggressive reclaim pays off.
> 

Hi Michal,

Fair enough. I have seen DB and cloud server workloads suffer under
default behavior of reclaim/compaction. It manifests itself as prolonged
delays in populating new database and in launching new cloud
applications. It is fair to ask for the predictive algorithm to be
proven before pulling something like this in kernel. I will implement
this same algorithm in userspace and use existing knobs to tune kernel
dynamically. Running that with large number of workloads will provide
data on how often does this help. If I find any useful tunables missing,
I will be sure to bring it up.

Thanks,
Khalid