| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20250814153307.1553061-1-raghavendra.kt@amd.com>
Date: Thu, 14 Aug 2025 15:32:50 +0000
From: Raghavendra K T <raghavendra.kt@....com>
To: <raghavendra.kt@....com>
CC: <AneeshKumar.KizhakeVeetil@....com>, <Michael.Day@....com>,
<akpm@...ux-foundation.org>, <bharata@....com>, <dave.hansen@...el.com>,
<david@...hat.com>, <dongjoo.linux.dev@...il.com>, <feng.tang@...el.com>,
<gourry@...rry.net>, <hannes@...xchg.org>, <honggyu.kim@...com>,
<hughd@...gle.com>, <jhubbard@...dia.com>, <jon.grimm@....com>,
<k.shutemov@...il.com>, <kbusch@...a.com>, <kmanaouil.dev@...il.com>,
<leesuyeon0506@...il.com>, <leillc@...gle.com>, <liam.howlett@...cle.com>,
<linux-kernel@...r.kernel.org>, <linux-mm@...ck.org>,
<mgorman@...hsingularity.net>, <mingo@...hat.com>, <nadav.amit@...il.com>,
<nphamcs@...il.com>, <peterz@...radead.org>, <riel@...riel.com>,
<rientjes@...gle.com>, <rppt@...nel.org>, <santosh.shukla@....com>,
<shivankg@....com>, <shy828301@...il.com>, <sj@...nel.org>, <vbabka@...e.cz>,
<weixugc@...gle.com>, <willy@...radead.org>, <ying.huang@...ux.alibaba.com>,
<ziy@...dia.com>, <Jonathan.Cameron@...wei.com>, <dave@...olabs.net>,
<yuanchu@...gle.com>, <kinseyho@...gle.com>, <hdanton@...a.com>,
<harry.yoo@...cle.com>
Subject: [RFC PATCH V3 00/17] mm: slowtier page promotion based on PTE A bit
The current series has additional enhancements and comments' incorporation on top of
RFC V2.
This is an additional source of hot page generator to NUMAB, IBS [4], KMGLRUD [5].
Introduction:
=============
In the current hot page promotion, all the activities including the
process address space scanning, NUMA hint fault handling and page
migration is performed in the process context. i.e., scanning overhead is
borne by applications.
This RFC V2 patch series does slow-tier page promotion by using PTE Accessed
bit scanning. Scanning is done by a global kernel thread which routinely
scans all the processes' address spaces and checks for accesses by reading
the PTE A bit.
A separate migration thread migrates/promotes the pages to the top-tier
node based on a simple heuristic that uses top-tier scan/access information
of the mm.
Additionally based on the feedback, a prctl knob with a scalar value is
provided to control per task scanning.
Changes Since RFC V2:
===================
- Enhanced logic to migrate on second access.
- Using prctl scalar value to further tune the scanning efficiency.
- Using of PFN instead of folio to record hot pages for easy integration
with kpromoted/kmigrated [4].
- Rebasing on top of fork/exec changes in v6.16.
- Revisiting mm_walk logic and folio validation based on Harry's comments.
- Feedback from migration system to slowdown scanning when more migration failures
happen.
- Masami's comment on trace patch.
- Bug fix to overnight idle system crash due to incorrect kmemcache usage.
- Enhanced target node finding logic to further obtain fallback nodes to migrate.
(TBD: This needs followup patch that actually does migration to fallback target nodes)
Changes since RFC V1:
=====================
- Addressing the review comments by Jonathan (Thank you for your closer
reviews).
- Per mm migration list with separate lock to resolve race conditions/softlockups
reported by Davidlohr.
- Add one more filter before migration for LRU_GEN case to check whether
folio is still hot.
- Rename kmmscand ==> kscand kmmmigrated ==> kmigrated (hopefully this
gets merged into Bharat's upcoming migration thread)
Changes since RFC V0:
======================
- A separate migration thread is used for migration, thus alleviating need for
multi-threaded scanning (at least as per tracing).
- A simple heuristic for target node calculation is added.
- prctl (David R) interface with scalar value is added to control per task scanning.
- Steve's comment on tracing incorporated.
- Davidlohr's reported bugfix.
- Initial scan delay similar to NUMAB1 mode added.
- Got rid of migration lock during mm_walk.
A note on per mm migration list using mm_slot:
=============================================
Using per mm migration list (mm_slot) has helped to reduce contention
and thus easing mm teardown during process exit.
It also helps to tie PFN/folio with mm to make heuristics work better
and further it would help to throttle migration per mm (OR process) (TBD).
A note on PTE A bit scanning:
============================
Major positive: Current patchset is able to cover all the process address
space scanning effectively with simple algorithms to tune scan_size and
scan_period.
Thanks to Jonathan, Davidlohr, David, Harry, Masami Steve for review feedback on RFCs.
Future plans:
================
Evaluate how integration with hotness monitoring subsystem works, OR
as a standalone integration with kmigrated API* of [4]
Results:
=======
Benchmark Cbench (by Bharata) to evaluate performance promotion in
slowtier system.
Benchmark allocates memory on both regular NUMA node and slowtier node,
then does continuous access.
Goal: Finishing fixed numaber of access in less time
SUT: Genoa+ EPYC system
base 6.16 NUMAB2 (because this has the best perf)
patched 6.16 + current series
Time taken in sec (lower is better)
base patched
8GB 228 206
32GB 547 534
128GB 1100 920
Links:
[1] RFC V0: https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@amd.com/
[2] RFC V1: https://lore.kernel.org/linux-mm/20250319193028.29514-1-raghavendra.kt@amd.com/
[3] RFC V2: https://lore.kernel.org/linux-mm/20250624055617.1291159-1-raghavendra.kt@amd.com/
[4] Hotpage detection and promotion: https://lore.kernel.org/linux-mm/20250814134826.154003-1-bharata@amd.com/T/#t
[5] MGLRU: https://lkml.org/lkml/2025/3/24/1458
Patch organization:
patch 1-5 initial skeleton for scanning and migration
patch 6: migration
patch 7-9: scanning optimizations
patch 10: target_node heuristic
patch 11: Migration failure feedback
patch 12-14: sysfs, vmstat and tracing
patch 15-16: prctl implementation and enhancements to scanning.
patch17: Fallback target node finding
Raghavendra K T (17):
mm: Add kscand kthread for PTE A bit scan
mm: Maintain mm_struct list in the system
mm: Scan the mm and create a migration list
mm/kscand: Add only hot pages to migration list
mm: Create a separate kthread for migration
mm/migration: migrate accessed folios to toptier node
mm: Add throttling of mm scanning using scan_period
mm: Add throttling of mm scanning using scan_size
mm: Add initial scan delay
mm: Add a heuristic to calculate target node
mm/kscand: Implement migration failure feedback
sysfs: Add sysfs support to tune scanning
mm/vmstat: Add vmstat counters
trace/kscand: Add tracing of scanning and migration
prctl: Introduce new prctl to control scanning
prctl: Fine tune scan_period with prctl scale param
mm: Create a list of fallback target nodes
Documentation/filesystems/proc.rst | 2 +
fs/proc/task_mmu.c | 4 +
include/linux/kscand.h | 30 +
include/linux/migrate.h | 2 +
include/linux/mm.h | 13 +
include/linux/mm_types.h | 7 +
include/linux/vm_event_item.h | 12 +
include/trace/events/kmem.h | 99 ++
include/uapi/linux/prctl.h | 7 +
kernel/fork.c | 6 +
kernel/sys.c | 25 +
mm/Kconfig | 8 +
mm/Makefile | 1 +
mm/internal.h | 1 +
mm/kscand.c | 1754 ++++++++++++++++++++++++++++
mm/migrate.c | 2 +-
mm/mmap.c | 2 +
mm/vma_exec.c | 3 +
mm/vmstat.c | 12 +
19 files changed, 1989 insertions(+), 1 deletion(-)
create mode 100644 include/linux/kscand.h
create mode 100644 mm/kscand.c
base-commit: 038d61fd642278bab63ee8ef722c50d10ab01e8f
--
2.34.1
Powered by blists - more mailing lists