lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Thu, 8 Aug 2013 12:18:07 +0200
From:	Jan Kara <jack@...e.cz>
To:	Andy Lutomirski <luto@...capital.net>
Cc:	Dave Hansen <dave.hansen@...el.com>, Jan Kara <jack@...e.cz>,
	linux-mm@...ck.org, linux-ext4@...r.kernel.org,
	linux-kernel@...r.kernel.org
Subject: Re: [RFC 0/3] Add madvise(..., MADV_WILLWRITE)

On Wed 07-08-13 11:00:52, Andy Lutomirski wrote:
> On Wed, Aug 7, 2013 at 10:40 AM, Dave Hansen <dave.hansen@...el.com> wrote:
> > On 08/07/2013 06:40 AM, Jan Kara wrote:
> >>   One question before I look at the patches: Why don't you use fallocate()
> >> in your application? The functionality you require seems to be pretty
> >> similar to it - writing to an already allocated block is usually quick.
> >
> > One problem I've seen is that it still costs you a fault per-page to get
> > the PTEs in to a state where you can write to the memory.  MADV_WILLNEED
> > will do readahead to get the page cache filled, but it still leaves the
> > pages unmapped.  Those faults get expensive when you're trying to do a
> > couple hundred million of them all at once.
> 
> I have grand plans to teach the kernel to use hardware dirty tracking
> so that (some?) pages can be left clean and writable for long periods
> of time.  This will be hard.
  Right that will be tough... Although with your application you could
require such pages to be mlocked and then I could imagine we would get away
at least from problems with dirty page accounting.

> Even so, the second write fault to a page tends to take only a few
> microseconds, while the first one often blocks in fs code.
  So you wrote blocks are already preallocated with fallocate(). If you
also preload pages in memory with MADV_WILLNEED is there still big
difference between the first and subsequent write fault?

> (mmap_sem is a different story, but I see it as a separate issue.)
  Yeah, agreed.

								Honza
-- 
Jan Kara <jack@...e.cz>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Powered by blists - more mailing lists