linux-kernel - [PATCH v1 07/14] mm/pagewalk.c: add pud_entry

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20210313075747.3781593-8-yuzhao@google.com>
Date:   Sat, 13 Mar 2021 00:57:40 -0700
From:   Yu Zhao <yuzhao@...gle.com>
To:     linux-mm@...ck.org
Cc:     Alex Shi <alex.shi@...ux.alibaba.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        Hillf Danton <hdanton@...a.com>,
        Johannes Weiner <hannes@...xchg.org>,
        Joonsoo Kim <iamjoonsoo.kim@....com>,
        Matthew Wilcox <willy@...radead.org>,
        Mel Gorman <mgorman@...e.de>, Michal Hocko <mhocko@...e.com>,
        Roman Gushchin <guro@...com>, Vlastimil Babka <vbabka@...e.cz>,
        Wei Yang <richard.weiyang@...ux.alibaba.com>,
        Yang Shi <shy828301@...il.com>,
        Ying Huang <ying.huang@...el.com>,
        linux-kernel@...r.kernel.org, page-reclaim@...gle.com,
        Yu Zhao <yuzhao@...gle.com>
Subject: [PATCH v1 07/14] mm/pagewalk.c: add pud_entry_post() for post-order traversals

Add a new callback pud_entry_post() to struct mm_walk_ops so that page
table walkers can visit the non-leaf PMD entries of a PUD entry after
they have visited with the leaf PTE entries. This allows page table
walkers who clear the accessed bit to take advantage of the last
commit, in a similar way walk_pte_range() works for the PTE entries of
a PMD entry: they only need to take PTL once to search all the child
entries of a parent entry.

Signed-off-by: Yu Zhao <yuzhao@...gle.com>
---
 include/linux/pagewalk.h | 4 ++++
 mm/pagewalk.c            | 5 +++++
 2 files changed, 9 insertions(+)

diff --git a/include/linux/pagewalk.h b/include/linux/pagewalk.h
index b1cb6b753abb..2b68ae9d27d3 100644
--- a/include/linux/pagewalk.h
+++ b/include/linux/pagewalk.h
@@ -11,6 +11,8 @@ struct mm_walk;
  * @pgd_entry:		if set, called for each non-empty PGD (top-level) entry
  * @p4d_entry:		if set, called for each non-empty P4D entry
  * @pud_entry:		if set, called for each non-empty PUD entry
+ * @pud_entry_post:	if set, called for each non-empty PUD entry after
+ *			pmd_entry is called, for post-order traversal.
  * @pmd_entry:		if set, called for each non-empty PMD entry
  *			this handler is required to be able to handle
  *			pmd_trans_huge() pmds.  They may simply choose to
@@ -41,6 +43,8 @@ struct mm_walk_ops {
 			 unsigned long next, struct mm_walk *walk);
 	int (*pud_entry)(pud_t *pud, unsigned long addr,
 			 unsigned long next, struct mm_walk *walk);
+	int (*pud_entry_post)(pud_t *pud, unsigned long addr,
+			      unsigned long next, struct mm_walk *walk);
 	int (*pmd_entry)(pmd_t *pmd, unsigned long addr,
 			 unsigned long next, struct mm_walk *walk);
 	int (*pte_entry)(pte_t *pte, unsigned long addr,
diff --git a/mm/pagewalk.c b/mm/pagewalk.c
index e81640d9f177..8ed1533f7eda 100644
--- a/mm/pagewalk.c
+++ b/mm/pagewalk.c
@@ -160,6 +160,11 @@ static int walk_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end,
 		err = walk_pmd_range(pud, addr, next, walk);
 		if (err)
 			break;
+
+		if (ops->pud_entry_post)
+			err = ops->pud_entry_post(pud, addr, next, walk);
+		if (err)
+			break;
 	} while (pud++, addr = next, addr != end);
 
 	return err;
-- 
2.31.0.rc2.261.g7f71774620-goog