Message-ID: <20241023170759.999909-1-surenb@google.com>
Date: Wed, 23 Oct 2024 10:07:53 -0700
From: Suren Baghdasaryan <surenb@...gle.com>
To: akpm@...ux-foundation.org
Cc: kent.overstreet@...ux.dev, corbet@....net, arnd@...db.de, 
	mcgrof@...nel.org, rppt@...nel.org, paulmck@...nel.org, thuth@...hat.com, 
	tglx@...utronix.de, bp@...en8.de, xiongwei.song@...driver.com, 
	ardb@...nel.org, david@...hat.com, vbabka@...e.cz, mhocko@...e.com, 
	hannes@...xchg.org, roman.gushchin@...ux.dev, dave@...olabs.net, 
	willy@...radead.org, liam.howlett@...cle.com, pasha.tatashin@...een.com, 
	souravpanda@...gle.com, keescook@...omium.org, dennis@...nel.org, 
	jhubbard@...dia.com, urezki@...il.com, hch@...radead.org, petr.pavlu@...e.com, 
	samitolvanen@...gle.com, da.gomez@...sung.com, yuzhao@...gle.com, 
	vvvvvv@...gle.com, rostedt@...dmis.org, iamjoonsoo.kim@....com, 
	rientjes@...gle.com, minchan@...gle.com, kaleshsingh@...gle.com, 
	linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org, 
	linux-arch@...r.kernel.org, linux-mm@...ck.org, 
	maple-tree@...ts.infradead.org, linux-modules@...r.kernel.org, 
	kernel-team@...roid.com, surenb@...gle.com
Subject: [PATCH v4 0/6] page allocation tag compression

This patchset implements several improvements:
1. Gracefully handles module unloading while there are used allocations
allocated from that module;
2. Provides an option to store page allocation tag references in the
page flags, removing dependency on page extensions and eliminating the
memory overhead from storing page allocation references (~0.2% of total
system memory). This also improves page allocation performance when
CONFIG_MEM_ALLOC_PROFILING is enabled by eliminating page extension
lookup. Page allocation performance overhead is reduced from 41% to 5.5%.

Patch #1 introduces mas_for_each_rev() helper function.

Patch #2 introduces shutdown_mem_profiling() helper function to be used
when disabling memory allocation profiling.

Patch #3 copies module tags into virtually contiguous memory which
serves two purposes:
- Lets us deal with the situation when module is unloaded while there
are still live allocations from that module. Since we are using a copy
version of the tags we can safely unload the module. Space and gaps in
this contiguous memory are managed using a maple tree.
- Enables simple indexing of the tags in the later patches.

Patch #4 changes the way we allocate virtually contiguous memory for
module tags to reserve only vitrual area and populate physical pages
only as needed at module load time.

Patch #5 abstracts page allocation tag reference to simplify later
changes.

Patch #6 adds compression option to the sysctl.vm.mem_profiling boot
parameter for storing page allocation tag references inside page flags
if they fit. If the number of available page flag bits is insufficient
to address all kernel allocations, memory allocation profiling gets
disabled with an appropriate warning.

Patchset applies to mm-unstable.

Changes since v3 [1]:
- rebased over Mike's patchset in mm-unstable
- added Reviewed-by, per Liam Howlett
- limited execmem_vmap to work with EXECMEM_MODULE_DATA only,
per Mike Rapoport
- moved __get_vm_area_node() declaration into mm/internal.h,
per Mike Rapoport
- split parts of reserve_module_tags() into helper functions to make it
more readable, per Mike Rapoport
- introduced shutdown_mem_profiling() to be used when disabling memory
allocation profiling
- replaced CONFIG_PGALLOC_TAG_USE_PAGEFLAGS with a new boot parameter
option, per Michal Hocko
- minor code cleanups and refactoring to make the code more readable
- added VMALLOC and MODULE SUPPORT reviewers I missed before

[1] https://lore.kernel.org/all/20241014203646.1952505-1-surenb@google.com/

Suren Baghdasaryan (6):
  maple_tree: add mas_for_each_rev() helper
  alloc_tag: introduce shutdown_mem_profiling helper function
  alloc_tag: load module tags into separate contiguous memory
  alloc_tag: populate memory for module tags as needed
  alloc_tag: introduce pgtag_ref_handle to abstract page tag references
  alloc_tag: support for page allocation tag compression

 Documentation/mm/allocation-profiling.rst |   7 +-
 include/asm-generic/codetag.lds.h         |  19 +
 include/linux/alloc_tag.h                 |  21 +-
 include/linux/codetag.h                   |  40 +-
 include/linux/execmem.h                   |  10 +
 include/linux/maple_tree.h                |  14 +
 include/linux/mm.h                        |  25 +-
 include/linux/page-flags-layout.h         |   7 +
 include/linux/pgalloc_tag.h               | 197 +++++++--
 include/linux/vmalloc.h                   |   3 +
 kernel/module/main.c                      |  80 ++--
 lib/alloc_tag.c                           | 467 ++++++++++++++++++++--
 lib/codetag.c                             | 104 ++++-
 mm/execmem.c                              |  16 +
 mm/internal.h                             |   6 +
 mm/mm_init.c                              |   5 +-
 mm/vmalloc.c                              |   4 +-
 scripts/module.lds.S                      |   5 +-
 18 files changed, 903 insertions(+), 127 deletions(-)


base-commit: b5d43fad926a3f542cd06f3c9d286f6f489f7129
-- 
2.47.0.105.g07ac214952-goog

Powered by blists - more mailing lists