linux-kernel - [PATCH v3 1/2] mm/ksm: support fork/exec for prctl

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 21 Sep 2023 09:47:08 -0700
From:   Stefan Roesch <shr@...kernel.io>
To:     kernel-team@...com
Cc:     shr@...kernel.io, akpm@...ux-foundation.org, david@...hat.com,
        hannes@...xchg.org, riel@...riel.com, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org, Carl Klemm <carl@...s.xyz>
Subject: [PATCH v3 1/2] mm/ksm: support fork/exec for prctl

A process can enable KSM with the prctl system call. When the process is
forked the KSM flag is inherited by the child process. However if the
process is executing an exec system call directly after the fork, the
KSM setting is cleared. This patch addresses this problem.

Signed-off-by: Stefan Roesch <shr@...kernel.io>
Fixes: d7597f59d1d3 ("mm: add new api to enable ksm per process")
Reviewed-by: David Hildenbrand <david@...hat.com>
Reported-by: Carl Klemm <carl@...s.xyz>
Tested-by: Carl Klemm <carl@...s.xyz>
---
 include/linux/sched/coredump.h | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/include/linux/sched/coredump.h b/include/linux/sched/coredump.h
index 0ee96ea7a0e9..205aa9917394 100644
--- a/include/linux/sched/coredump.h
+++ b/include/linux/sched/coredump.h
@@ -87,8 +87,11 @@ static inline int get_dumpable(struct mm_struct *mm)
 
 #define MMF_DISABLE_THP_MASK	(1 << MMF_DISABLE_THP)
 
+#define MMF_VM_MERGE_ANY	29
+#define MMF_VM_MERGE_ANY_MASK	(1 << MMF_VM_MERGE_ANY)
+
 #define MMF_INIT_MASK		(MMF_DUMPABLE_MASK | MMF_DUMP_FILTER_MASK |\
-				 MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK)
+				 MMF_DISABLE_THP_MASK | MMF_HAS_MDWE_MASK |\
+				 MMF_VM_MERGE_ANY_MASK)
 
-#define MMF_VM_MERGE_ANY	29
 #endif /* _LINUX_SCHED_COREDUMP_H */
-- 
2.39.3