linux-kernel - [PATCH] mm: optimize migrate_vma

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20201021191335.10916-1-rcampbell@nvidia.com>
Date:   Wed, 21 Oct 2020 12:13:35 -0700
From:   Ralph Campbell <rcampbell@...dia.com>
To:     <linux-mm@...ck.org>, <linux-kernel@...r.kernel.org>
CC:     Jerome Glisse <jglisse@...hat.com>,
        John Hubbard <jhubbard@...dia.com>,
        Alistair Popple <apopple@...dia.com>,
        Christoph Hellwig <hch@....de>,
        "Jason Gunthorpe" <jgg@...dia.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "Ralph Campbell" <rcampbell@...dia.com>
Subject: [PATCH] mm: optimize migrate_vma_pages() mmu notifier

When migrating a zero page or pte_none() anonymous page to device private
memory, migrate_vma_setup() will initialize the src[] array with a NULL
PFN. This lets the device driver allocate device private memory and clear
it instead of DMAing a page of zeros over the device bus. Since the source
page didn't exist at the time, no struct page was locked nor a migration
PTE inserted into the CPU page tables. The actual PTE insertion happens
in migrate_vma_pages() when it tries to insert the device private struct
page PTE into the CPU page tables. migrate_vma_pages() has to call the
mmu notifiers again since another device could fault on the same page
before the page table locks are acquired. Allow device drivers to optimize
the invalidation similar to migrate_vma_setup() by calling
mmu_notifier_range_init() which sets struct mmu_notifier_range event type
to MMU_NOTIFY_MIGRATE and the migrate_pgmap_owner field.

Signed-off-by: Ralph Campbell <rcampbell@...dia.com>
---

This is for Andrew Morton's mm tree after the merge window.

 mm/migrate.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 5ca5842df5db..560b57dde960 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2999,11 +2999,10 @@ void migrate_vma_pages(struct migrate_vma *migrate)
 			if (!notified) {
 				notified = true;

-				mmu_notifier_range_init(&range,
-							MMU_NOTIFY_CLEAR, 0,
-							NULL,
-							migrate->vma->vm_mm,
-							addr, migrate->end);
+				mmu_notifier_range_init_migrate(&range, 0,
+					migrate->vma, migrate->vma->vm_mm,
+					addr, migrate->end,
+					migrate->pgmap_owner);
 				mmu_notifier_invalidate_range_start(&range);
 			}
 			migrate_vma_insert_page(migrate, addr, newpage,
-- 
2.20.1