| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 12 Dec 2012 17:42:53 +0900
From: Minchan Kim <minchan@...nel.org>
To: Wanpeng Li <liwanp@...ux.vnet.ibm.com>
Cc: John Stultz <john.stultz@...aro.org>,
Andrew Morton <akpm@...ux-foundation.org>, linux-mm@...ck.org,
linux-kernel@...r.kernel.org,
Michael Kerrisk <mtk.manpages@...il.com>,
Arun Sharma <asharma@...com>, sanjay@...gle.com,
Paul Turner <pjt@...gle.com>,
David Rientjes <rientjes@...gle.com>,
Christoph Lameter <cl@...ux.com>,
Android Kernel Team <kernel-team@...roid.com>,
Robert Love <rlove@...gle.com>, Mel Gorman <mel@....ul.ie>,
Hugh Dickins <hughd@...gle.com>,
Dave Hansen <dave@...ux.vnet.ibm.com>,
Rik van Riel <riel@...hat.com>,
Dave Chinner <david@...morbit.com>, Neil Brown <neilb@...e.de>,
Mike Hommey <mh@...ndium.org>, Taras Glek <tglek@...illa.com>,
KOSAKI Motohiro <kosaki.motohiro@...il.com>,
KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
Subject: Re: [RFC v3] Support volatile range for anon vma
On Wed, Dec 12, 2012 at 04:17:14PM +0800, Wanpeng Li wrote:
> On Wed, Dec 12, 2012 at 02:43:49PM +0800, Wanpeng Li wrote:
> >On Tue, Dec 11, 2012 at 11:41:04AM +0900, Minchan Kim wrote:
> >>Sorry, resending with fixing compile error. :(
> >>
> >>>From 0cfd3b65e4e90ab59abe8a337334414f92423cad Mon Sep 17 00:00:00 2001
> >>From: Minchan Kim <minchan@...nel.org>
> >>Date: Tue, 11 Dec 2012 11:38:30 +0900
> >>Subject: [RFC v3] Support volatile range for anon vma
> >>
> >>This still is [RFC v3] because just passed my simple test
> >>with TCMalloc tweaking.
> >>
> >>I hope more inputs from user-space allocator people and test patch
> >>with their allocator because it might need design change of arena
> >>management design for getting real vaule.
> >>
> >>Changelog from v2
> >>
> >> * Removing madvise(addr, length, MADV_NOVOLATILE).
> >> * add vmstat about the number of discarded volatile pages
> >> * discard volatile pages without promotion in reclaim path
> >>
> >>This is based on v3.6.
> >>
> >>- What's the madvise(addr, length, MADV_VOLATILE)?
> >>
> >> It's a hint that user deliver to kernel so kernel can *discard*
> >> pages in a range anytime.
> >>
> >>- What happens if user access page(ie, virtual address) discarded
> >> by kernel?
> >>
> >> The user can see zero-fill-on-demand pages as if madvise(DONTNEED).
> >>
> >>- What happens if user access page(ie, virtual address) doesn't
> >> discarded by kernel?
> >>
> >> The user can see old data without page fault.
> >>
> >>- What's different with madvise(DONTNEED)?
> >>
> >> System call semantic
> >>
> >> DONTNEED makes sure user always can see zero-fill pages after
> >> he calls madvise while VOLATILE can see zero-fill pages or
> >> old data.
> >>
> >> Internal implementation
> >>
> >> The madvise(DONTNEED) should zap all mapped pages in range so
> >> overhead is increased linearly with the number of mapped pages.
> >> Even, if user access zapped pages by write, page fault + page
> >> allocation + memset should be happened.
> >>
> >> The madvise(VOLATILE) should mark the flag in a range(ie, VMA).
> >> It doesn't touch pages any more so overhead of the system call
> >> should be very small. If memory pressure happens, VM can discard
> >> pages in VMAs marked by VOLATILE. If user access address with
> >> write mode by discarding by VM, he can see zero-fill pages so the
> >> cost is same with DONTNEED but if memory pressure isn't severe,
> >> user can see old data without (page fault + page allocation + memset)
> >>
> >> The VOLATILE mark should be removed in page fault handler when first
> >> page fault occur in marked vma so next page faults will follow normal
> >> page fault path. That's why user don't need madvise(MADV_NOVOLATILE)
> >> interface.
> >>
> >>- What's the benefit compared to DONTNEED?
> >>
> >> 1. The system call overhead is smaller because VOLATILE just marks
> >> the flag to VMA instead of zapping all the page in a range.
> >>
> >> 2. It has a chance to eliminate overheads (ex, page fault +
> >> page allocation + memset(PAGE_SIZE)).
> >>
> >>- Isn't there any drawback?
> >>
> >> DONTNEED doesn't need exclusive mmap_sem locking so concurrent page
> >> fault of other threads could be allowed. But VOLATILE needs exclusive
> >> mmap_sem so other thread would be blocked if they try to access
> >> not-mapped pages. That's why I designed madvise(VOLATILE)'s overhead
> >> should be small as far as possible.
> >>
> >> Other concern of exclusive mmap_sem is when page fault occur in
> >> VOLATILE marked vma. We should remove the flag of vma and merge
> >> adjacent vmas so needs exclusive mmap_sem. It can slow down page fault
> >> handling and prevent concurrent page fault. But we need such handling
> >> just once when page fault occur after we mark VOLATILE into VMA
> >> only if memory pressure happpens so the page is discarded. So it wouldn't
> >> not common so that benefit we get by this feature would be bigger than
> >> lose.
> >>
> >>- What's for targetting?
> >>
> >> Firstly, user-space allocator like ptmalloc, tcmalloc or heap management
> >> of virtual machine like Dalvik. Also, it comes in handy for embedded
> >> which doesn't have swap device so they can't reclaim anonymous pages.
> >> By discarding instead of swap, it could be used in the non-swap system.
> >> For it, we have to age anon lru list although we don't have swap because
> >> I don't want to discard volatile pages by top priority when memory pressure
> >> happens as volatile in this patch means "We don't need to swap out because
> >> user can handle the situation which data are disappear suddenly", NOT
> >> "They are useless so hurry up to reclaim them". So I want to apply same
> >> aging rule of nomal pages to them.
> >>
> >> Anonymous page background aging of non-swap system would be a trade-off
> >> for getting good feature. Even, we had done it two years ago until merge
> >> [1] and I believe gain of this patch will beat loss of anon lru aging's
> >> overead once all of allocator start to use madvise.
> >> (This patch doesn't include background aging in case of non-swap system
> >> but it's trivial if we decide)
> >>
> >>[1] 74e3f3c3, vmscan: prevent background aging of anon page in no swap system
> >>
> >>Cc: Michael Kerrisk <mtk.manpages@...il.com>
> >>Cc: Arun Sharma <asharma@...com>
> >>Cc: sanjay@...gle.com
> >>Cc: Paul Turner <pjt@...gle.com>
> >>CC: David Rientjes <rientjes@...gle.com>
> >>Cc: John Stultz <john.stultz@...aro.org>
> >>Cc: Andrew Morton <akpm@...ux-foundation.org>
> >>Cc: Christoph Lameter <cl@...ux.com>
> >>Cc: Android Kernel Team <kernel-team@...roid.com>
> >>Cc: Robert Love <rlove@...gle.com>
> >>Cc: Mel Gorman <mel@....ul.ie>
> >>Cc: Hugh Dickins <hughd@...gle.com>
> >>Cc: Dave Hansen <dave@...ux.vnet.ibm.com>
> >>Cc: Rik van Riel <riel@...hat.com>
> >>Cc: Dave Chinner <david@...morbit.com>
> >>Cc: Neil Brown <neilb@...e.de>
> >>Cc: Mike Hommey <mh@...ndium.org>
> >>Cc: Taras Glek <tglek@...illa.com>
> >>Cc: KOSAKI Motohiro <kosaki.motohiro@...il.com>
> >>Cc: Christoph Lameter <cl@...ux.com>
> >>Cc: KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
> >>Signed-off-by: Minchan Kim <minchan@...nel.org>
> >>---
> >> arch/x86/mm/fault.c | 2 +
> >> include/asm-generic/mman-common.h | 6 ++
> >> include/linux/mm.h | 7 ++-
> >> include/linux/rmap.h | 20 ++++++
> >> include/linux/vm_event_item.h | 2 +-
> >> mm/madvise.c | 19 +++++-
> >> mm/memory.c | 32 ++++++++++
> >> mm/migrate.c | 6 +-
> >> mm/rmap.c | 125 ++++++++++++++++++++++++++++++++++++-
> >> mm/vmscan.c | 7 +++
> >> mm/vmstat.c | 1 +
> >> 11 files changed, 218 insertions(+), 9 deletions(-)
> >>
> >>diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> >>index 76dcd9d..17c1c20 100644
> >>--- a/arch/x86/mm/fault.c
> >>+++ b/arch/x86/mm/fault.c
> >>@@ -879,6 +879,8 @@ mm_fault_error(struct pt_regs *regs, unsigned long error_code,
> >> }
> >>
> >> out_of_memory(regs, error_code, address);
> >>+ } else if (fault & VM_FAULT_SIGSEG) {
> >>+ bad_area(regs, error_code, address);
> >> } else {
> >> if (fault & (VM_FAULT_SIGBUS|VM_FAULT_HWPOISON|
> >> VM_FAULT_HWPOISON_LARGE))
> >>diff --git a/include/asm-generic/mman-common.h b/include/asm-generic/mman-common.h
> >>index d030d2c..f07781e 100644
> >>--- a/include/asm-generic/mman-common.h
> >>+++ b/include/asm-generic/mman-common.h
> >>@@ -34,6 +34,12 @@
> >> #define MADV_SEQUENTIAL 2 /* expect sequential page references */
> >> #define MADV_WILLNEED 3 /* will need these pages */
> >> #define MADV_DONTNEED 4 /* don't need these pages */
> >>+/*
> >>+ * Unlike other flags, we need two locks to protect MADV_VOLATILE.
> >>+ * For changing the flag, we need mmap_sem's write lock and volatile_lock
> >>+ * while we just need volatile_lock in case of reading the flag.
> >>+ */
> >>+#define MADV_VOLATILE 5 /* pages will disappear suddenly */
> >>
> >> /* common parameters: try to keep these consistent across architectures */
> >> #define MADV_REMOVE 9 /* remove these pages & resources */
> >>diff --git a/include/linux/mm.h b/include/linux/mm.h
> >>index 311be90..89027b5 100644
> >>--- a/include/linux/mm.h
> >>+++ b/include/linux/mm.h
> >>@@ -119,6 +119,7 @@ extern unsigned int kobjsize(const void *objp);
> >> #define VM_SAO 0x20000000 /* Strong Access Ordering (powerpc) */
> >> #define VM_PFN_AT_MMAP 0x40000000 /* PFNMAP vma that is fully mapped at mmap time */
> >> #define VM_MERGEABLE 0x80000000 /* KSM may merge identical pages */
> >>+#define VM_VOLATILE 0x100000000 /* Pages in the vma could be discarable without swap */
> >>
> >> /* Bits set in the VMA until the stack is in its final location */
> >> #define VM_STACK_INCOMPLETE_SETUP (VM_RAND_READ | VM_SEQ_READ)
> >>@@ -143,7 +144,7 @@ extern unsigned int kobjsize(const void *objp);
> >> * Special vmas that are non-mergable, non-mlock()able.
> >> * Note: mm/huge_memory.c VM_NO_THP depends on this definition.
> >> */
> >>-#define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_RESERVED | VM_PFNMAP)
> >>+#define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_RESERVED | VM_PFNMAP | VM_VOLATILE)
> >>
> >> /*
> >> * mapping from the currently active vm_flags protection bits (the
> >>@@ -872,11 +873,11 @@ static inline int page_mapped(struct page *page)
> >> #define VM_FAULT_NOPAGE 0x0100 /* ->fault installed the pte, not return page */
> >> #define VM_FAULT_LOCKED 0x0200 /* ->fault locked the returned page */
> >> #define VM_FAULT_RETRY 0x0400 /* ->fault blocked, must retry */
> >>-
> >>+#define VM_FAULT_SIGSEG 0x0800 /* -> There is no vma */
> >> #define VM_FAULT_HWPOISON_LARGE_MASK 0xf000 /* encodes hpage index for large hwpoison */
> >>
> >> #define VM_FAULT_ERROR (VM_FAULT_OOM | VM_FAULT_SIGBUS | VM_FAULT_HWPOISON | \
> >>- VM_FAULT_HWPOISON_LARGE)
> >>+ VM_FAULT_HWPOISON_LARGE | VM_FAULT_SIGSEG)
> >>
> >> /* Encode hstate index for a hwpoisoned large page */
> >> #define VM_FAULT_SET_HINDEX(x) ((x) << 12)
> >>diff --git a/include/linux/rmap.h b/include/linux/rmap.h
> >>index 3fce545..735d7a3 100644
> >>--- a/include/linux/rmap.h
> >>+++ b/include/linux/rmap.h
> >>@@ -67,6 +67,9 @@ struct anon_vma_chain {
> >> struct list_head same_anon_vma; /* locked by anon_vma->mutex */
> >> };
> >>
> >>+void volatile_lock(struct vm_area_struct *vma);
> >>+void volatile_unlock(struct vm_area_struct *vma);
> >>+
> >> #ifdef CONFIG_MMU
> >> static inline void get_anon_vma(struct anon_vma *anon_vma)
> >> {
> >>@@ -170,6 +173,7 @@ enum ttu_flags {
> >> TTU_IGNORE_MLOCK = (1 << 8), /* ignore mlock */
> >> TTU_IGNORE_ACCESS = (1 << 9), /* don't age */
> >> TTU_IGNORE_HWPOISON = (1 << 10),/* corrupted page is recoverable */
> >>+ TTU_IGNORE_VOLATILE = (1 << 11),/* ignore volatile */
> >> };
> >> #define TTU_ACTION(x) ((x) & TTU_ACTION_MASK)
> >>
> >>@@ -194,6 +198,21 @@ static inline pte_t *page_check_address(struct page *page, struct mm_struct *mm,
> >> return ptep;
> >> }
> >>
> >>+pte_t *__page_check_volatile_address(struct page *, struct mm_struct *,
> >>+ unsigned long, spinlock_t **);
> >>+
> >>+static inline pte_t *page_check_volatile_address(struct page *page,
> >>+ struct mm_struct *mm,
> >>+ unsigned long address,
> >>+ spinlock_t **ptlp)
> >>+{
> >>+ pte_t *ptep;
> >>+
> >>+ __cond_lock(*ptlp, ptep = __page_check_volatile_address(page,
> >>+ mm, address, ptlp));
> >>+ return ptep;
> >>+}
> >>+
> >> /*
> >> * Used by swapoff to help locate where page is expected in vma.
> >> */
> >>@@ -257,5 +276,6 @@ static inline int page_mkclean(struct page *page)
> >> #define SWAP_AGAIN 1
> >> #define SWAP_FAIL 2
> >> #define SWAP_MLOCK 3
> >>+#define SWAP_DISCARD 4
> >>
> >> #endif /* _LINUX_RMAP_H */
> >>diff --git a/include/linux/vm_event_item.h b/include/linux/vm_event_item.h
> >>index 57f7b10..3f9a40b 100644
> >>--- a/include/linux/vm_event_item.h
> >>+++ b/include/linux/vm_event_item.h
> >>@@ -23,7 +23,7 @@
> >>
> >> enum vm_event_item { PGPGIN, PGPGOUT, PSWPIN, PSWPOUT,
> >> FOR_ALL_ZONES(PGALLOC),
> >>- PGFREE, PGACTIVATE, PGDEACTIVATE,
> >>+ PGFREE, PGVOLATILE, PGACTIVATE, PGDEACTIVATE,
> >> PGFAULT, PGMAJFAULT,
> >> FOR_ALL_ZONES(PGREFILL),
> >> FOR_ALL_ZONES(PGSTEAL_KSWAPD),
> >>diff --git a/mm/madvise.c b/mm/madvise.c
> >>index 14d260f..53a19d8 100644
> >>--- a/mm/madvise.c
> >>+++ b/mm/madvise.c
> >>@@ -86,6 +86,13 @@ static long madvise_behavior(struct vm_area_struct * vma,
> >> if (error)
> >> goto out;
> >> break;
> >>+ case MADV_VOLATILE:
> >>+ if (vma->vm_flags & VM_LOCKED) {
> >>+ error = -EINVAL;
> >>+ goto out;
> >>+ }
> >>+ new_flags |= VM_VOLATILE;
> >>+ break;
> >> }
> >>
> >> if (new_flags == vma->vm_flags) {
> >>@@ -118,9 +125,13 @@ static long madvise_behavior(struct vm_area_struct * vma,
> >> success:
> >> /*
> >> * vm_flags is protected by the mmap_sem held in write mode.
> >>+ * In caes of MADV_VOLATILE, we need anon_vma_lock additionally.
> >> */
> >>+ if (behavior == MADV_VOLATILE)
> >>+ volatile_lock(vma);
> >> vma->vm_flags = new_flags;
> >>-
> >>+ if (behavior == MADV_VOLATILE)
> >>+ volatile_unlock(vma);
> >> out:
> >> if (error == -ENOMEM)
> >> error = -EAGAIN;
> >>@@ -310,6 +321,7 @@ madvise_behavior_valid(int behavior)
> >> #endif
> >> case MADV_DONTDUMP:
> >> case MADV_DODUMP:
> >>+ case MADV_VOLATILE:
> >> return 1;
> >>
> >> default:
> >>@@ -385,6 +397,11 @@ SYSCALL_DEFINE3(madvise, unsigned long, start, size_t, len_in, int, behavior)
> >> goto out;
> >> len = (len_in + ~PAGE_MASK) & PAGE_MASK;
> >>
> >>+ if (behavior != MADV_VOLATILE)
> >>+ len = (len_in + ~PAGE_MASK) & PAGE_MASK;
> >>+ else
> >>+ len = len_in & PAGE_MASK;
> >>+
> >> /* Check to see whether len was rounded up from small -ve to zero */
> >> if (len_in && !len)
> >> goto out;
> >>diff --git a/mm/memory.c b/mm/memory.c
> >>index 5736170..b5e4996 100644
> >>--- a/mm/memory.c
> >>+++ b/mm/memory.c
> >>@@ -57,6 +57,7 @@
> >> #include <linux/swapops.h>
> >> #include <linux/elf.h>
> >> #include <linux/gfp.h>
> >>+#include <linux/mempolicy.h>
> >>
> >> #include <asm/io.h>
> >> #include <asm/pgalloc.h>
> >>@@ -3446,6 +3447,37 @@ int handle_pte_fault(struct mm_struct *mm,
> >> return do_linear_fault(mm, vma, address,
> >> pte, pmd, flags, entry);
> >> }
> >>+ if (vma->vm_flags & VM_VOLATILE) {
> >>+ struct vm_area_struct *prev;
> >>+
> >>+ up_read(&mm->mmap_sem);
> >>+ down_write(&mm->mmap_sem);
> >>+ vma = find_vma_prev(mm, address, &prev);
> >>+
> >>+ /* Someone unmap the vma */
> >>+ if (unlikely(!vma) || vma->vm_start > address) {
> >>+ downgrade_write(&mm->mmap_sem);
> >>+ return VM_FAULT_SIGSEG;
> >>+ }
> >>+ /* Someone else already hanlded */
> >>+ if (vma->vm_flags & VM_VOLATILE) {
> >>+ /*
> >>+ * From now on, we hold mmap_sem as
> >>+ * exclusive.
> >>+ */
> >>+ volatile_lock(vma);
> >>+ vma->vm_flags &= ~VM_VOLATILE;
> >>+ volatile_unlock(vma);
> >>+
> >>+ vma_merge(mm, prev, vma->vm_start,
> >>+ vma->vm_end, vma->vm_flags,
> >>+ vma->anon_vma, vma->vm_file,
> >>+ vma->vm_pgoff, vma_policy(vma));
> >>+
> >>+ }
> >>+
> >>+ downgrade_write(&mm->mmap_sem);
> >>+ }
> >> return do_anonymous_page(mm, vma, address,
> >> pte, pmd, flags);
> >> }
> >>diff --git a/mm/migrate.c b/mm/migrate.c
> >>index 77ed2d7..08b009c 100644
> >>--- a/mm/migrate.c
> >>+++ b/mm/migrate.c
> >>@@ -800,7 +800,8 @@ static int __unmap_and_move(struct page *page, struct page *newpage,
> >> }
> >>
> >> /* Establish migration ptes or remove ptes */
> >>- try_to_unmap(page, TTU_MIGRATION|TTU_IGNORE_MLOCK|TTU_IGNORE_ACCESS);
> >>+ try_to_unmap(page, TTU_MIGRATION|TTU_IGNORE_MLOCK|
> >>+ TTU_IGNORE_ACCESS|TTU_IGNORE_VOLATILE);
> >>
> >> skip_unmap:
> >> if (!page_mapped(page))
> >>@@ -915,7 +916,8 @@ static int unmap_and_move_huge_page(new_page_t get_new_page,
> >> if (PageAnon(hpage))
> >> anon_vma = page_get_anon_vma(hpage);
> >>
> >>- try_to_unmap(hpage, TTU_MIGRATION|TTU_IGNORE_MLOCK|TTU_IGNORE_ACCESS);
> >>+ try_to_unmap(hpage, TTU_MIGRATION|TTU_IGNORE_MLOCK|
> >>+ TTU_IGNORE_ACCESS|TTU_IGNORE_VOLATILE);
> >>
> >> if (!page_mapped(hpage))
> >> rc = move_to_new_page(new_hpage, hpage, 1, mode);
> >>diff --git a/mm/rmap.c b/mm/rmap.c
> >>index 0f3b7cd..1a0ab2b 100644
> >>--- a/mm/rmap.c
> >>+++ b/mm/rmap.c
> >>@@ -603,6 +603,57 @@ unsigned long page_address_in_vma(struct page *page, struct vm_area_struct *vma)
> >> return vma_address(page, vma);
> >> }
> >>
> >>+pte_t *__page_check_volatile_address(struct page *page, struct mm_struct *mm,
> >>+ unsigned long address, spinlock_t **ptlp)
> >>+{
> >>+ pgd_t *pgd;
> >>+ pud_t *pud;
> >>+ pmd_t *pmd;
> >>+ pte_t *pte;
> >>+ spinlock_t *ptl;
> >>+
> >>+ swp_entry_t entry = { .val = page_private(page) };
> >>+
> >>+ if (unlikely(PageHuge(page))) {
> >>+ pte = huge_pte_offset(mm, address);
> >>+ ptl = &mm->page_table_lock;
> >>+ goto check;
> >>+ }
> >>+
> >>+ pgd = pgd_offset(mm, address);
> >>+ if (!pgd_present(*pgd))
> >>+ return NULL;
> >>+
> >>+ pud = pud_offset(pgd, address);
> >>+ if (!pud_present(*pud))
> >>+ return NULL;
> >>+
> >>+ pmd = pmd_offset(pud, address);
> >>+ if (!pmd_present(*pmd))
> >>+ return NULL;
> >>+ if (pmd_trans_huge(*pmd))
> >>+ return NULL;
> >>+
> >>+ pte = pte_offset_map(pmd, address);
> >>+ ptl = pte_lockptr(mm, pmd);
> >>+check:
> >>+ spin_lock(ptl);
> >>+ if (PageAnon(page)) {
> >>+ if (!pte_present(*pte) && entry.val ==
> >>+ pte_to_swp_entry(*pte).val) {
> >>+ *ptlp = ptl;
> >>+ return pte;
> >>+ }
> >>+ } else {
> >>+ if (pte_none(*pte)) {
> >>+ *ptlp = ptl;
> >>+ return pte;
> >>+ }
> >>+ }
> >>+ pte_unmap_unlock(pte, ptl);
> >>+ return NULL;
> >>+}
> >>+
> >> /*
> >> * Check that @page is mapped at @address into @mm.
> >> *
> >>@@ -1218,6 +1269,35 @@ out:
> >> mem_cgroup_end_update_page_stat(page, &locked, &flags);
> >> }
> >>
> >>+int try_to_zap_one(struct page *page, struct vm_area_struct *vma,
> >>+ unsigned long address)
> >>+{
> >>+ struct mm_struct *mm = vma->vm_mm;
> >>+ pte_t *pte;
> >>+ pte_t pteval;
> >>+ spinlock_t *ptl;
> >>+
> >>+ pte = page_check_volatile_address(page, mm, address, &ptl);
> >>+ if (!pte)
> >>+ return 0;
> >>+
> >>+ /* Nuke the page table entry. */
> >>+ flush_cache_page(vma, address, page_to_pfn(page));
> >>+ pteval = ptep_clear_flush(vma, address, pte);
> >>+
> >>+ if (PageAnon(page)) {
> >>+ swp_entry_t entry = { .val = page_private(page) };
> >>+ if (PageSwapCache(page)) {
> >>+ dec_mm_counter(mm, MM_SWAPENTS);
> >>+ swap_free(entry);
> >>+ }
> >>+ }
> >>+
> >>+ pte_unmap_unlock(pte, ptl);
> >>+ mmu_notifier_invalidate_page(mm, address);
> >>+ return 1;
> >>+}
> >>+
> >> /*
> >> * Subfunctions of try_to_unmap: try_to_unmap_one called
> >> * repeatedly from try_to_unmap_ksm, try_to_unmap_anon or try_to_unmap_file.
> >>@@ -1494,6 +1574,10 @@ static int try_to_unmap_anon(struct page *page, enum ttu_flags flags)
> >> struct anon_vma *anon_vma;
> >> struct anon_vma_chain *avc;
> >> int ret = SWAP_AGAIN;
> >>+ bool is_volatile = true;
> >>+
> >>+ if (flags & TTU_IGNORE_VOLATILE)
> >>+ is_volatile = false;
> >>
> >> anon_vma = page_lock_anon_vma(page);
> >> if (!anon_vma)
> >>@@ -1512,17 +1596,40 @@ static int try_to_unmap_anon(struct page *page, enum ttu_flags flags)
> >> * temporary VMAs until after exec() completes.
> >> */
> >> if (IS_ENABLED(CONFIG_MIGRATION) && (flags & TTU_MIGRATION) &&
> >>- is_vma_temporary_stack(vma))
> >>+ is_vma_temporary_stack(vma)) {
> >>+ is_volatile = false;
> >> continue;
> >>+ }
> >>
> >> address = vma_address(page, vma);
> >> if (address == -EFAULT)
> >> continue;
> >>+ /*
> >>+ * A volatile page will only be purged if ALL vmas
> >>+ * pointing to it are VM_VOLATILE.
> >>+ */
> >>+ if (!(vma->vm_flags & VM_VOLATILE))
> >>+ is_volatile = false;
> >>+
> >> ret = try_to_unmap_one(page, vma, address, flags);
> >> if (ret != SWAP_AGAIN || !page_mapped(page))
> >> break;
> >> }
> >>
> >>+ if (page_mapped(page) || is_volatile == false)
> >>+ goto out;
> >>+
> >>+ list_for_each_entry(avc, &anon_vma->head, same_anon_vma) {
> >>+ struct vm_area_struct *vma = avc->vma;
> >>+ unsigned long address;
> >>+
> >>+ address = vma_address(page, vma);
> >>+ try_to_zap_one(page, vma, address);
> >>+ }
> >>+ /* We're throwing this page out, so mark it clean */
> >>+ ClearPageDirty(page);
> >>+ ret = SWAP_DISCARD;
> >>+out:
> >> page_unlock_anon_vma(anon_vma);
> >> return ret;
> >> }
> >>@@ -1651,6 +1758,7 @@ out:
> >> * SWAP_AGAIN - we missed a mapping, try again later
> >> * SWAP_FAIL - the page is unswappable
> >> * SWAP_MLOCK - page is mlocked.
> >>+ * SWAP_DISCARD - page is volatile.
> >> */
> >> int try_to_unmap(struct page *page, enum ttu_flags flags)
> >> {
> >>@@ -1665,7 +1773,8 @@ int try_to_unmap(struct page *page, enum ttu_flags flags)
> >> ret = try_to_unmap_anon(page, flags);
> >> else
> >> ret = try_to_unmap_file(page, flags);
> >>- if (ret != SWAP_MLOCK && !page_mapped(page))
> >>+ if (ret != SWAP_MLOCK && !page_mapped(page) &&
> >>+ ret != SWAP_DISCARD)
> >> ret = SWAP_SUCCESS;
> >> return ret;
> >> }
> >>@@ -1707,6 +1816,18 @@ void __put_anon_vma(struct anon_vma *anon_vma)
> >> anon_vma_free(anon_vma);
> >> }
> >>
> >>+void volatile_lock(struct vm_area_struct *vma)
> >>+{
> >>+ if (vma->anon_vma)
> >>+ anon_vma_lock(vma->anon_vma);
> >>+}
> >>+
> >>+void volatile_unlock(struct vm_area_struct *vma)
> >>+{
> >>+ if (vma->anon_vma)
> >>+ anon_vma_unlock(vma->anon_vma);
> >>+}
> >>+
> >> #ifdef CONFIG_MIGRATION
> >> /*
> >> * rmap_walk() and its helpers rmap_walk_anon() and rmap_walk_file():
> >>diff --git a/mm/vmscan.c b/mm/vmscan.c
> >>index 99b434b..4e463a4 100644
> >>--- a/mm/vmscan.c
> >>+++ b/mm/vmscan.c
> >>@@ -630,6 +630,9 @@ static enum page_references page_check_references(struct page *page,
> >> if (vm_flags & VM_LOCKED)
> >> return PAGEREF_RECLAIM;
> >>
> >>+ if (vm_flags & VM_VOLATILE)
> >>+ return PAGEREF_RECLAIM;
> >>+
> >> if (referenced_ptes) {
> >> if (PageSwapBacked(page))
> >> return PAGEREF_ACTIVATE;
> >>@@ -789,6 +792,9 @@ static unsigned long shrink_page_list(struct list_head *page_list,
> >> */
> >
> >Hi Minchan,
> >
> >IIUC, anonymous page has already add to swapcache through add_to_swap called
> >by shrink_page_list, but I can't figure out where you remove it from swapache.
> >
>
> Yeah, they all done in shrink_page_list, I mean if you can avoid the process of
> add to swapcache and remove it from swapcache since your idea don't need swapout.
It's not simple because wee need swap entry for unmapping to store it in PTEs
since we can't expect it's volatile or not. In addition, we have to put the page
into swapcache before puting it to pte so it can handle race with another fault
for swapin.
So If it's not a severe problem, I want to leave just as it is, which is so simple.
--
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists