linux-kernel - Re: [PATCH 0/2] mm: tlb swap entries batch async release

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGsJ_4xdnQjQyJVMfN7ZSW3OMvJhFRErjwMGSCDZACQOVWeesw@mail.gmail.com>
Date: Wed, 31 Jul 2024 10:18:49 +0800
From: Barry Song <21cnbao@...il.com>
To: Zhiguo Jiang <justinjiang@...o.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org, 
	linux-kernel@...r.kernel.org, Will Deacon <will@...nel.org>, 
	"Aneesh Kumar K.V" <aneesh.kumar@...nel.org>, Nick Piggin <npiggin@...il.com>, 
	Peter Zijlstra <peterz@...radead.org>, Arnd Bergmann <arnd@...db.de>, 
	Johannes Weiner <hannes@...xchg.org>, Michal Hocko <mhocko@...nel.org>, 
	Roman Gushchin <roman.gushchin@...ux.dev>, Shakeel Butt <shakeel.butt@...ux.dev>, 
	Muchun Song <muchun.song@...ux.dev>, linux-arch@...r.kernel.org, cgroups@...r.kernel.org, 
	opensource.kernel@...o.com
Subject: Re: [PATCH 0/2] mm: tlb swap entries batch async release

On Tue, Jul 30, 2024 at 7:44 PM Zhiguo Jiang <justinjiang@...o.com> wrote:
>
> The main reasons for the prolonged exit of a background process is the
> time-consuming release of its swap entries. The proportion of swap memory
> occupied by the background process increases with its duration in the
> background, and after a period of time, this value can reach 60% or more.

Do you know the reason? Could they be contending for a cluster lock or
something?
Is there any perf data or flamegraph available here?

> Additionally, the relatively lengthy path for releasing swap entries
> further contributes to the longer time required for the background process
> to release its swap entries.
>
> In the multiple background applications scenario, when launching a large
> memory application such as a camera, system may enter a low memory state,
> which will triggers the killing of multiple background processes at the
> same time. Due to multiple exiting processes occupying multiple CPUs for
> concurrent execution, the current foreground application's CPU resources
> are tight and may cause issues such as lagging.
>
> To solve this problem, we have introduced the multiple exiting process
> asynchronous swap memory release mechanism, which isolates and caches
> swap entries occupied by multiple exit processes, and hands them over
> to an asynchronous kworker to complete the release. This allows the
> exiting processes to complete quickly and release CPU resources. We have
> validated this modification on the products and achieved the expected
> benefits.
>
> It offers several benefits:
> 1. Alleviate the high system cpu load caused by multiple exiting
>    processes running simultaneously.
> 2. Reduce lock competition in swap entry free path by an asynchronous

 Do you have data on which lock is affected? Could it be a cluster lock?

>    kworker instead of multiple exiting processes parallel execution.
> 3. Release memory occupied by exiting processes more efficiently.
>
> Zhiguo Jiang (2):
>   mm: move task_is_dying to h headfile
>   mm: tlb: multiple exiting processes's swap entries async release
>
>  include/asm-generic/tlb.h |  50 +++++++
>  include/linux/mm_types.h  |  58 ++++++++
>  include/linux/oom.h       |   6 +
>  mm/memcontrol.c           |   6 -
>  mm/memory.c               |   3 +-
>  mm/mmu_gather.c           | 297 ++++++++++++++++++++++++++++++++++++++
>  6 files changed, 413 insertions(+), 7 deletions(-)
>  mode change 100644 => 100755 include/asm-generic/tlb.h
>  mode change 100644 => 100755 include/linux/mm_types.h
>  mode change 100644 => 100755 include/linux/oom.h
>  mode change 100644 => 100755 mm/memcontrol.c
>  mode change 100644 => 100755 mm/memory.c
>  mode change 100644 => 100755 mm/mmu_gather.c

Can you check your local filesystem to determine why you're running
the chmod command?

>
> --
> 2.39.0
>

Thanks
Barry