linux-kernel - Re: [PATCH] lazy freeing of memory through MADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <462C8E1D.8000706@redhat.com>
Date:	Mon, 23 Apr 2007 06:44:45 -0400
From:	Rik van Riel <riel@...hat.com>
To:	Nick Piggin <nickpiggin@...oo.com.au>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel <linux-kernel@...r.kernel.org>,
	linux-mm <linux-mm@...ck.org>, shak <dshaks@...hat.com>,
	jakub@...hat.com, drepper@...hat.com
Subject: Re: [PATCH] lazy freeing of memory through MADV_FREE

Use TLB batching for MADV_FREE.  Adds another 10-15% extra performance
to the MySQL sysbench results on my quad core system.

Signed-off-by: Rik van Riel <riel@...hat.com>
---

Nick Piggin wrote:

>> 3) because of this, we can treat any such accesses as
>>    happening simultaneously with the MADV_FREE and
>>    as illegal, aka undefined behaviour territory and
>>    we do not need to worry about them
> 
> Yes, but I'm wondering if it is legal in all architectures.

It's similar to trying to access memory during an munmap.

You may be able to for a short time, but it'll come back to
haunt you.

>> 4) because we flush the tlb before releasing the page
>>    table lock, other CPUs cannot remove this page from
>>    the address space - they will block on the page
>>    table lock before looking at this pte
> 
> We don't when the ptl is split.

Even then we do.  Each invocation of zap_pte_range() only touches
one page table page, and it flushes the TLB before releasing the
page table lock.

> What the tlb flush used to be able to assume is that the page
> has been removed from the pagetables when they are put in the
> tlb flush batch.

All the tlb flush code seems to assume is that the tlb entries
should be invalidated.

> I'm not saying there is any bugs, but just suggesting there
> might be.

Jakub found a potential bug, in that I did not use an atomic
operation to clear the page table entries.  I've attached a
new patch which simply uses ptep_test_and_clear_dirty/young
to get rid of the dirty and accessed bits.

It uses the same atomic accesses we use elsewhere in the VM
and the code is a line shorter than before.

Andrew, please use this one.

-- 
Politics is the struggle between those who want to make their country
the best in the world, and those who believe it already is.  Each group
calls the other unpatriotic.

View attachment "linux-2.6-madv_free-lazytlb.patch" of type "text/x-patch" (698 bytes)