linux-kernel - Re: [PATCH] thp: reduce usage of huge zero page's atomic counter

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <36b76a95-5025-ac64-0862-b98b2ebdeaf7@intel.com>
Date:   Tue, 30 Aug 2016 11:09:15 +0800
From:   Aaron Lu <aaron.lu@...el.com>
To:     Andrew Morton <akpm@...ux-foundation.org>
Cc:     Linux Memory Management List <linux-mm@...ck.org>,
        "'Kirill A. Shutemov'" <kirill.shutemov@...ux.intel.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        Huang Ying <ying.huang@...el.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Jerome Marchand <jmarchan@...hat.com>,
        Andrea Arcangeli <aarcange@...hat.com>,
        Mel Gorman <mgorman@...hsingularity.net>,
        Ebru Akagunduz <ebru.akagunduz@...il.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH] thp: reduce usage of huge zero page's atomic counter

On 08/30/2016 06:50 AM, Andrew Morton wrote:
> On Mon, 29 Aug 2016 14:31:20 +0800 Aaron Lu <aaron.lu@...el.com> wrote:
> 
>>
>> The global zero page is used to satisfy an anonymous read fault. If
>> THP(Transparent HugePage) is enabled then the global huge zero page is used.
>> The global huge zero page uses an atomic counter for reference counting
>> and is allocated/freed dynamically according to its counter value.
>>
>> CPU time spent on that counter will greatly increase if there are
>> a lot of processes doing anonymous read faults. This patch proposes a
>> way to reduce the access to the global counter so that the CPU load
>> can be reduced accordingly.
>>
>> To do this, a new flag of the mm_struct is introduced: MMF_USED_HUGE_ZERO_PAGE.
>> With this flag, the process only need to touch the global counter in
>> two cases:
>> 1 The first time it uses the global huge zero page;
>> 2 The time when mm_user of its mm_struct reaches zero.
>>
>> Note that right now, the huge zero page is eligible to be freed as soon
>> as its last use goes away.  With this patch, the page will not be
>> eligible to be freed until the exit of the last process from which it
>> was ever used.
>>
>> And with the use of mm_user, the kthread is not eligible to use huge
>> zero page either. Since no kthread is using huge zero page today, there
>> is no difference after applying this patch. But if that is not desired,
>> I can change it to when mm_count reaches zero.
> 
> I suppose we could simply never free the zero huge page - if some
> process has used it in the past, others will probably use it in the
> future.  One wonders how useful this optimization is...
>
> But the patch is simple enough.
> 
>> Case used for test on Haswell EP:
>> usemem -n 72 --readonly -j 0x200000 100G
>> Which spawns 72 processes and each will mmap 100G anonymous space and
>> then do read only access to that space sequentially with a step of 2MB.
>>
>> perf report for base commit:
>>     54.03%  usemem   [kernel.kallsyms]   [k] get_huge_zero_page
>> perf report for this commit:
>>      0.11%  usemem   [kernel.kallsyms]   [k] mm_get_huge_zero_page
> 
> Does this mean that overall usemem runtime halved?

Sorry for the confusion, the above line is extracted from perf report.
It shows the percent of CPU cycles executed in a specific function.

The above two perf lines are used to show get_huge_zero_page doesn't
consume that much CPU cycles after applying the patch.

> 
> Do we have any numbers for something which is more real-wordly?

Unfortunately, no real world numbers.

We think the global atomic counter could be an issue for performance
so I'm trying to solve the problem.

Thanks,
Aaron