linux-kernel - Re: [NFS] blocks of zeros (NULLs) in NFS files in kernels >= 2.6.20

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20080922170414.GC12483@merfinllc.com>
Date:	Mon, 22 Sep 2008 10:04:14 -0700
From:	Aaron Straus <aaron@...finllc.com>
To:	Trond Myklebust <trond.myklebust@....uio.no>
Cc:	Hans-Peter Jansen <hpj@...la.net>, linux-kernel@...r.kernel.org,
	Chuck Lever <chuck.lever@...cle.com>,
	Neil Brown <neilb@...e.de>,
	Linux NFS Mailing List <linux-nfs@...r.kernel.org>
Subject: Re: [NFS] blocks of zeros (NULLs) in NFS files in kernels >= 2.6.20

Hi,

On Sep 22 12:35 PM, Trond Myklebust wrote:
> Revert _what_ exactly?

Yep.  I narrowed the problem down to an offending hunk in a particular
patch.  Removing that hunk did eliminate the problem.   However,
reverting that hunk is likely wrong and the code has changed _a lot_
since that commit.

> My understanding was that this is a consequence of unordered writes
> causing the file to be extended while some other task is reading.

Yes.  I added some debugging statements to look at the writeout path.  

I think the following happens:

  - page 0 gets partially written to by app
  - VM writes out partial dirty page 0
  - page 0 gets fully written by app
  - page 1 gets partially written by app
  - VM writes out dirty page 1

  At this point there is a hole in the file.  The tail end of page 0 is
still not written to server.

  - VM writes out dirty page 0
  ...

> AFAICS, this sort of behaviour has _always_ been possible. I can't see
> how reverting anything will fix it.

Here is the crux.  It was possible previously but unlikely e.g. our app
never saw this behavior.  The new writeout semantics causes visible
holes in files often.

Anyway, I agree the new writeout semantics are allowed and possibly
saner than the previous writeout path.  The problem is that it is
__annoying__ for this use case (log files).

I'm not sure if there is an easy solution.  We want the VM to writeout
the address space in order.   Maybe we can start the scan for dirty
pages at the last page we wrote out i.e. page 0 in the example above?

					=a=

-- 
===================
Aaron Straus
aaron@...finllc.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/