linux-kernel - Re: [PATCH 0/2] [RFC] Volatile ranges (v4)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <loom.20120719T113441-155@post.gmane.org>
Date:	Thu, 19 Jul 2012 10:13:41 +0000 (UTC)
From:	Dmitry Vyukov <dvyukov@...gle.com>
To:	linux-kernel@...r.kernel.org
Subject: Re: [PATCH 0/2] [RFC] Volatile ranges (v4)

John Stultz <john.stultz <at> linaro.org> writes:
> Ok. So here's another iteration of the fadvise volatile range code.
> 
> I realize this is still a way off from being ready, but I wanted to post
> what I have to share with folks working on the various range/interval
> management ideas as well as update folks who've provided feedback on the
> volatile range code.

Hi John,

May you please confirm whether the fadvise(FADV_VOLATILE)
will work for me in the following case?
I am developing a dynamic data race detector (ThreadSanitizer).
It needs to associate some meta-data (shadow) with each byte
of application memory (4 bytes of shadow for 1 byte of app 
memory). We mmap(MAP_ANONYMOUS|MAP_NORESERVE)
40TB of virtual address space for 
shadow, and then just access it as needed. It works.
But for some apps that consume too much 
memory, it eventually leads to swapping/OOM kills.
There is a property of shadow memory that I would like to exploit
 - any region of shadow memory can be reset to zero at any point
w/o any bad consequences (it can lead to missed data 
races, but it's better than OOM kill).
I've tried to execute madvise(MADV_DONTNEED) every X 
seconds for whole shadow memory. It almost works.
The problem is that madvise() seems to be 
not atomic, occasionally I see missed writes, that's not acceptable,
I need either zero pages or 
consistent pages.
Your FADV_VOLATILE looks like it may be the solution.
To summarize: I have a huge region of memory that
I would like to mark as "volatile" at program 
startup. The region is anonymous (not backed by any file).
The kernel must be able to take away 
any pages in the range, and then return zero pages later.
I do have concurrent writes to the 
range, and missed writes are unacceptable.
Ideally, pages are revoked on LRU basis
TIA

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/