| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20260114134510.1835-2-kalyazin@amazon.com>
Date: Wed, 14 Jan 2026 13:45:23 +0000
From: "Kalyazin, Nikita" <kalyazin@...zon.co.uk>
To: "kvm@...r.kernel.org" <kvm@...r.kernel.org>, "linux-doc@...r.kernel.org"
<linux-doc@...r.kernel.org>, "linux-kernel@...r.kernel.org"
<linux-kernel@...r.kernel.org>, "linux-arm-kernel@...ts.infradead.org"
<linux-arm-kernel@...ts.infradead.org>, "kvmarm@...ts.linux.dev"
<kvmarm@...ts.linux.dev>, "linux-fsdevel@...r.kernel.org"
<linux-fsdevel@...r.kernel.org>, "linux-mm@...ck.org" <linux-mm@...ck.org>,
"bpf@...r.kernel.org" <bpf@...r.kernel.org>,
"linux-kselftest@...r.kernel.org" <linux-kselftest@...r.kernel.org>,
"kernel@...0n.name" <kernel@...0n.name>, "linux-riscv@...ts.infradead.org"
<linux-riscv@...ts.infradead.org>, "linux-s390@...r.kernel.org"
<linux-s390@...r.kernel.org>, "loongarch@...ts.linux.dev"
<loongarch@...ts.linux.dev>
CC: "pbonzini@...hat.com" <pbonzini@...hat.com>, "corbet@....net"
<corbet@....net>, "maz@...nel.org" <maz@...nel.org>, "oupton@...nel.org"
<oupton@...nel.org>, "joey.gouly@....com" <joey.gouly@....com>,
"suzuki.poulose@....com" <suzuki.poulose@....com>, "yuzenghui@...wei.com"
<yuzenghui@...wei.com>, "catalin.marinas@....com" <catalin.marinas@....com>,
"will@...nel.org" <will@...nel.org>, "seanjc@...gle.com" <seanjc@...gle.com>,
"tglx@...utronix.de" <tglx@...utronix.de>, "mingo@...hat.com"
<mingo@...hat.com>, "bp@...en8.de" <bp@...en8.de>,
"dave.hansen@...ux.intel.com" <dave.hansen@...ux.intel.com>, "x86@...nel.org"
<x86@...nel.org>, "hpa@...or.com" <hpa@...or.com>, "luto@...nel.org"
<luto@...nel.org>, "peterz@...radead.org" <peterz@...radead.org>,
"willy@...radead.org" <willy@...radead.org>, "akpm@...ux-foundation.org"
<akpm@...ux-foundation.org>, "david@...nel.org" <david@...nel.org>,
"lorenzo.stoakes@...cle.com" <lorenzo.stoakes@...cle.com>,
"Liam.Howlett@...cle.com" <Liam.Howlett@...cle.com>, "vbabka@...e.cz"
<vbabka@...e.cz>, "rppt@...nel.org" <rppt@...nel.org>, "surenb@...gle.com"
<surenb@...gle.com>, "mhocko@...e.com" <mhocko@...e.com>, "ast@...nel.org"
<ast@...nel.org>, "daniel@...earbox.net" <daniel@...earbox.net>,
"andrii@...nel.org" <andrii@...nel.org>, "martin.lau@...ux.dev"
<martin.lau@...ux.dev>, "eddyz87@...il.com" <eddyz87@...il.com>,
"song@...nel.org" <song@...nel.org>, "yonghong.song@...ux.dev"
<yonghong.song@...ux.dev>, "john.fastabend@...il.com"
<john.fastabend@...il.com>, "kpsingh@...nel.org" <kpsingh@...nel.org>,
"sdf@...ichev.me" <sdf@...ichev.me>, "haoluo@...gle.com" <haoluo@...gle.com>,
"jolsa@...nel.org" <jolsa@...nel.org>, "jgg@...pe.ca" <jgg@...pe.ca>,
"jhubbard@...dia.com" <jhubbard@...dia.com>, "peterx@...hat.com"
<peterx@...hat.com>, "jannh@...gle.com" <jannh@...gle.com>,
"pfalcato@...e.de" <pfalcato@...e.de>, "shuah@...nel.org" <shuah@...nel.org>,
"riel@...riel.com" <riel@...riel.com>, "ryan.roberts@....com"
<ryan.roberts@....com>, "jgross@...e.com" <jgross@...e.com>,
"yu-cheng.yu@...el.com" <yu-cheng.yu@...el.com>, "kas@...nel.org"
<kas@...nel.org>, "coxu@...hat.com" <coxu@...hat.com>,
"kevin.brodsky@....com" <kevin.brodsky@....com>, "ackerleytng@...gle.com"
<ackerleytng@...gle.com>, "maobibo@...ngson.cn" <maobibo@...ngson.cn>,
"prsampat@....com" <prsampat@....com>, "mlevitsk@...hat.com"
<mlevitsk@...hat.com>, "jmattson@...gle.com" <jmattson@...gle.com>,
"jthoughton@...gle.com" <jthoughton@...gle.com>, "agordeev@...ux.ibm.com"
<agordeev@...ux.ibm.com>, "alex@...ti.fr" <alex@...ti.fr>,
"aou@...s.berkeley.edu" <aou@...s.berkeley.edu>, "borntraeger@...ux.ibm.com"
<borntraeger@...ux.ibm.com>, "chenhuacai@...nel.org" <chenhuacai@...nel.org>,
"dev.jain@....com" <dev.jain@....com>, "gor@...ux.ibm.com"
<gor@...ux.ibm.com>, "hca@...ux.ibm.com" <hca@...ux.ibm.com>,
"Jonathan.Cameron@...wei.com" <Jonathan.Cameron@...wei.com>,
"palmer@...belt.com" <palmer@...belt.com>, "pjw@...nel.org" <pjw@...nel.org>,
"shijie@...amperecomputing.com" <shijie@...amperecomputing.com>,
"svens@...ux.ibm.com" <svens@...ux.ibm.com>, "thuth@...hat.com"
<thuth@...hat.com>, "wyihan@...gle.com" <wyihan@...gle.com>,
"yang@...amperecomputing.com" <yang@...amperecomputing.com>,
"vannapurve@...gle.com" <vannapurve@...gle.com>, "jackmanb@...gle.com"
<jackmanb@...gle.com>, "aneesh.kumar@...nel.org" <aneesh.kumar@...nel.org>,
"patrick.roy@...ux.dev" <patrick.roy@...ux.dev>, "Thomson, Jack"
<jackabt@...zon.co.uk>, "Itazuri, Takahiro" <itazur@...zon.co.uk>,
"Manwaring, Derek" <derekmn@...zon.com>, "Cali, Marco"
<xmarcalx@...zon.co.uk>, "Kalyazin, Nikita" <kalyazin@...zon.co.uk>
Subject: [PATCH v9 01/13] set_memory: add folio_{zap,restore}_direct_map
helpers
From: Nikita Kalyazin <kalyazin@...zon.com>
These allow guest_memfd to remove its memory from the direct map.
Only implement them for architectures that have direct map.
In folio_zap_direct_map(), flush TLB on architectures where
set_direct_map_valid_noflush() does not flush it internally.
The new helpers need to be accessible to KVM on architectures that
support guest_memfd (x86 and arm64). Since arm64 does not support
building KVM as a module, only export them on x86.
Direct map removal gives guest_memfd the same protection that
memfd_secret does, such as hardening against Spectre-like attacks
through in-kernel gadgets.
Signed-off-by: Nikita Kalyazin <kalyazin@...zon.com>
---
arch/arm64/include/asm/set_memory.h | 2 ++
arch/arm64/mm/pageattr.c | 12 ++++++++++++
arch/loongarch/include/asm/set_memory.h | 2 ++
arch/loongarch/mm/pageattr.c | 16 ++++++++++++++++
arch/riscv/include/asm/set_memory.h | 2 ++
arch/riscv/mm/pageattr.c | 16 ++++++++++++++++
arch/s390/include/asm/set_memory.h | 2 ++
arch/s390/mm/pageattr.c | 18 ++++++++++++++++++
arch/x86/include/asm/set_memory.h | 2 ++
arch/x86/mm/pat/set_memory.c | 20 ++++++++++++++++++++
include/linux/set_memory.h | 10 ++++++++++
11 files changed, 102 insertions(+)
diff --git a/arch/arm64/include/asm/set_memory.h b/arch/arm64/include/asm/set_memory.h
index 90f61b17275e..d949f1deb701 100644
--- a/arch/arm64/include/asm/set_memory.h
+++ b/arch/arm64/include/asm/set_memory.h
@@ -14,6 +14,8 @@ int set_memory_valid(unsigned long addr, int numpages, int enable);
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_default_noflush(struct page *page);
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
+int folio_zap_direct_map(struct folio *folio);
+int folio_restore_direct_map(struct folio *folio);
bool kernel_page_present(struct page *page);
int set_memory_encrypted(unsigned long addr, int numpages);
diff --git a/arch/arm64/mm/pageattr.c b/arch/arm64/mm/pageattr.c
index f0e784b963e6..a94eff324dda 100644
--- a/arch/arm64/mm/pageattr.c
+++ b/arch/arm64/mm/pageattr.c
@@ -357,6 +357,18 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
return set_memory_valid(addr, nr, valid);
}
+int folio_zap_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), false);
+}
+
+int folio_restore_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), true);
+}
+
#ifdef CONFIG_DEBUG_PAGEALLOC
/*
* This is - apart from the return value - doing the same
diff --git a/arch/loongarch/include/asm/set_memory.h b/arch/loongarch/include/asm/set_memory.h
index 55dfaefd02c8..9bc80ac420a9 100644
--- a/arch/loongarch/include/asm/set_memory.h
+++ b/arch/loongarch/include/asm/set_memory.h
@@ -18,5 +18,7 @@ bool kernel_page_present(struct page *page);
int set_direct_map_default_noflush(struct page *page);
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
+int folio_zap_direct_map(struct folio *folio);
+int folio_restore_direct_map(struct folio *folio);
#endif /* _ASM_LOONGARCH_SET_MEMORY_H */
diff --git a/arch/loongarch/mm/pageattr.c b/arch/loongarch/mm/pageattr.c
index f5e910b68229..14bd322dd112 100644
--- a/arch/loongarch/mm/pageattr.c
+++ b/arch/loongarch/mm/pageattr.c
@@ -236,3 +236,19 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
return __set_memory(addr, 1, set, clear);
}
+
+int folio_zap_direct_map(struct folio *folio)
+{
+ int ret;
+
+ ret = set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), false);
+
+ return ret;
+}
+
+int folio_restore_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), true);
+}
diff --git a/arch/riscv/include/asm/set_memory.h b/arch/riscv/include/asm/set_memory.h
index 87389e93325a..16557b70c830 100644
--- a/arch/riscv/include/asm/set_memory.h
+++ b/arch/riscv/include/asm/set_memory.h
@@ -43,6 +43,8 @@ static inline int set_kernel_memory(char *startp, char *endp,
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_default_noflush(struct page *page);
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
+int folio_zap_direct_map(struct folio *folio);
+int folio_restore_direct_map(struct folio *folio);
bool kernel_page_present(struct page *page);
#endif /* __ASSEMBLER__ */
diff --git a/arch/riscv/mm/pageattr.c b/arch/riscv/mm/pageattr.c
index 3f76db3d2769..2c218868114b 100644
--- a/arch/riscv/mm/pageattr.c
+++ b/arch/riscv/mm/pageattr.c
@@ -401,6 +401,22 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
return __set_memory((unsigned long)page_address(page), nr, set, clear);
}
+int folio_zap_direct_map(struct folio *folio)
+{
+ int ret;
+
+ ret = set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), false);
+
+ return ret;
+}
+
+int folio_restore_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), true);
+}
+
#ifdef CONFIG_DEBUG_PAGEALLOC
static int debug_pagealloc_set_page(pte_t *pte, unsigned long addr, void *data)
{
diff --git a/arch/s390/include/asm/set_memory.h b/arch/s390/include/asm/set_memory.h
index 94092f4ae764..fc73652e5715 100644
--- a/arch/s390/include/asm/set_memory.h
+++ b/arch/s390/include/asm/set_memory.h
@@ -63,6 +63,8 @@ __SET_MEMORY_FUNC(set_memory_4k, SET_MEMORY_4K)
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_default_noflush(struct page *page);
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
+int folio_zap_direct_map(struct folio *folio);
+int folio_restore_direct_map(struct folio *folio);
bool kernel_page_present(struct page *page);
#endif
diff --git a/arch/s390/mm/pageattr.c b/arch/s390/mm/pageattr.c
index d3ce04a4b248..df4a487b484d 100644
--- a/arch/s390/mm/pageattr.c
+++ b/arch/s390/mm/pageattr.c
@@ -412,6 +412,24 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
return __set_memory((unsigned long)page_to_virt(page), nr, flags);
}
+int folio_zap_direct_map(struct folio *folio)
+{
+ unsigned long addr = (unsigned long)folio_address(folio);
+ int ret;
+
+ ret = set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), false);
+ flush_tlb_kernel_range(addr, addr + folio_size(folio));
+
+ return ret;
+}
+
+int folio_restore_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), true);
+}
+
bool kernel_page_present(struct page *page)
{
unsigned long addr;
diff --git a/arch/x86/include/asm/set_memory.h b/arch/x86/include/asm/set_memory.h
index 61f56cdaccb5..7208af609121 100644
--- a/arch/x86/include/asm/set_memory.h
+++ b/arch/x86/include/asm/set_memory.h
@@ -90,6 +90,8 @@ int set_pages_rw(struct page *page, int numpages);
int set_direct_map_invalid_noflush(struct page *page);
int set_direct_map_default_noflush(struct page *page);
int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid);
+int folio_zap_direct_map(struct folio *folio);
+int folio_restore_direct_map(struct folio *folio);
bool kernel_page_present(struct page *page);
extern int kernel_set_to_readonly;
diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c
index 6c6eb486f7a6..3f0fc30eb320 100644
--- a/arch/x86/mm/pat/set_memory.c
+++ b/arch/x86/mm/pat/set_memory.c
@@ -2656,6 +2656,26 @@ int set_direct_map_valid_noflush(struct page *page, unsigned nr, bool valid)
return __set_pages_np(page, nr);
}
+int folio_zap_direct_map(struct folio *folio)
+{
+ unsigned long addr = (unsigned long)folio_address(folio);
+ int ret;
+
+ ret = set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), false);
+ flush_tlb_kernel_range(addr, addr + folio_size(folio));
+
+ return ret;
+}
+EXPORT_SYMBOL_FOR_MODULES(folio_zap_direct_map, "kvm");
+
+int folio_restore_direct_map(struct folio *folio)
+{
+ return set_direct_map_valid_noflush(folio_page(folio, 0),
+ folio_nr_pages(folio), true);
+}
+EXPORT_SYMBOL_FOR_MODULES(folio_restore_direct_map, "kvm");
+
#ifdef CONFIG_DEBUG_PAGEALLOC
void __kernel_map_pages(struct page *page, int numpages, int enable)
{
diff --git a/include/linux/set_memory.h b/include/linux/set_memory.h
index 3030d9245f5a..8d1c8a7f7d79 100644
--- a/include/linux/set_memory.h
+++ b/include/linux/set_memory.h
@@ -40,6 +40,16 @@ static inline int set_direct_map_valid_noflush(struct page *page,
return 0;
}
+static inline int folio_zap_direct_map(struct folio *folio)
+{
+ return 0;
+}
+
+static inline int folio_restore_direct_map(struct folio *folio)
+{
+ return 0;
+}
+
static inline bool kernel_page_present(struct page *page)
{
return true;
--
2.50.1
Powered by blists - more mailing lists