linux-kernel - Re: [PATCH v5 00/12] MADV

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <20160207123120.GA16116@bbox>
Date:	Sun, 7 Feb 2016 21:31:20 +0900
From:	Minchan Kim <minchan@...nel.org>
To:	"Michael Kerrisk (man-pages)" <mtk.manpages@...il.com>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	linux-api@...r.kernel.org, Hugh Dickins <hughd@...gle.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	Jason Evans <je@...com>, Daniel Micay <danielmicay@...il.com>,
	"Kirill A. Shutemov" <kirill@...temov.name>,
	Shaohua Li <shli@...nel.org>, Michal Hocko <mhocko@...e.cz>,
	yalin.wang2010@...il.com, Andy Lutomirski <luto@...capital.net>
Subject: Re: [PATCH v5 00/12] MADV_FREE support

On Sat, Feb 06, 2016 at 02:32:02PM +0100, Michael Kerrisk (man-pages) wrote:
> Hello Minchan,
> 
> On 02/05/2016 03:15 AM, Minchan Kim wrote:
> > On Thu, Jan 28, 2016 at 08:16:25AM +0100, Michael Kerrisk (man-pages) wrote:
> >> Hello Minchan,
> >>
> >> On 11/30/2015 07:39 AM, Minchan Kim wrote:
> >>> In v4, Andrew wanted to settle in old basic MADV_FREE and introduces
> >>> new stuffs(ie, lazyfree LRU, swapless support and lazyfreeness) later
> >>> so this version doesn't include them.
> >>>
> >>> I have been tested it on mmotm-2015-11-25-17-08 with additional
> >>> patch[1] from Kirill to prevent BUG_ON which he didn't send to
> >>> linux-mm yet as formal patch. With it, I couldn't find any
> >>> problem so far.
> >>>
> >>> Note that this version is based on THP refcount redesign so
> >>> I needed some modification on MADV_FREE because split_huge_pmd
> >>> doesn't split a THP page any more and pmd_trans_huge(pmd) is not
> >>> enough to guarantee the page is not THP page.
> >>> As well, for MAVD_FREE lazy-split, THP split should respect
> >>> pmd's dirtiness rather than marking ptes of all subpages dirty
> >>> unconditionally. Please, review last patch in this patchset.
> >>
> >> Now that MADV_FREE has been merged, would you be willing to write
> >> patch to the madvise(2) man page that describes the semantics, 
> >> noes limitations and restrictions, and (ideally) has some sentences
> >> describing use cases?
> >>
> > 
> > Hello Michael,
> > 
> > Could you review this patch?
> > 
> > Thanks.
> > 
> >>From 203372f901f574e991215fdff6907608ba53f932 Mon Sep 17 00:00:00 2001
> > From: Minchan Kim <minchan@...nel.org>
> > Date: Fri, 5 Feb 2016 11:09:54 +0900
> > Subject: [PATCH] madvise.2: Add MADV_FREE
> > 
> > Document the MADV_FREE flags added to madvise() in Linux 4.5
> > 
> > Signed-off-by: Minchan Kim <minchan@...nel.org>
> > ---
> >  man2/madvise.2 | 19 +++++++++++++++++++
> >  1 file changed, 19 insertions(+)
> > 
> > diff --git a/man2/madvise.2 b/man2/madvise.2
> > index c1df67c..4704304 100644
> > --- a/man2/madvise.2
> > +++ b/man2/madvise.2
> > @@ -143,6 +143,25 @@ flag are special memory areas that are not managed
> >  by the virtual memory subsystem.
> >  Such pages are typically created by device drivers that
> >  map the pages into user space.)
> > +.TP
> > +.B MADV_FREE " (since Linux 4.5)"
> > +Application is finished with the given range, so kernel can free
> > +resources associated with it but the freeing could be delayed until
> > +memory pressure happens or canceld by write operation by user.
> > +
> > +After a successful MADV_FREE operation, user shouldn't expect kernel
> > +keeps stale data on the page. However, subsequent write of pages
> > +in the range will succeed and then kernel cannot free those dirtied pages
> > +so user can always see just written data. If there was no subsequent
> > +write, kernel can free those clean pages any time. In such case,
> > +user can see zero-fill-on-demand pages.
> > +
> > +Note that, it works only with private anonymous pages (see
> > +.BR mmap (2)).
> > +On swapless system, freeing pages in given range happens instantly
> > +regardless of memory pressure.
> > +
> > +
> >  .\"
> >  .\" ======================================================================
> >  .\"
> > 
> 
> Thanks for the nice text! I reworked somewhat, trying to fill out a
> few details about how I understand things work, but I may have introduced
> errors, so I would be happy if you would check the following text:

Below looks good to me.
Thanks, Michael

> 
>        MADV_FREE (since Linux 4.5)
>               The  application  no  longer  requires  the pages in the
>               range specified by addr and len.  The  kernel  can  thus
>               free these pages, but the freeing could be delayed until
>               memory pressure occurs.  For each of the pages that  has
>               been  marked to be freed but has not yet been freed, the
>               free operation will be canceled  if  the  caller  writes
>               into  the page.  After a successful MADV_FREE operation,
>               any stale data (i.e., dirty, unwritten  pages)  will  be
>               lost  when  the kernel frees the pages.  However, subse‐
>               quent writes to pages in the range will succeed and then
>               kernel  cannot  free  those  dirtied  pages, so that the
>               caller can always see just written data.  If there is no
>               subsequent  write,  the kernel can free the pages at any
>               time.  Once pages in the  range  have  been  freed,  the
>               caller  will  see  zero-fill-on-demand pages upon subse‐
>               quent page references.
> 
>               The MADV_FREE operation can be applied only  to  private
>               anonymous  pages  (see  mmap(2)).  On a swapless system,
>               freeing  pages  in  a  given  range  happens  instantly,
>               regardless of memory pressure.
> 
> Thanks,
> 
> Michael
> 
> -- 
> Michael Kerrisk
> Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
> Linux/UNIX System Programming Training: http://man7.org/training/
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@...ck.org"> email@...ck.org </a>