linux-ext4 - Re: OpenPosix test case mmap_11-4 fails in ext4 filesystem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20140521141203.GB1501@thunk.org>
Date:	Wed, 21 May 2014 10:12:03 -0400
From:	Theodore Ts'o <tytso@....edu>
To:	Lukáš Czerner <lczerner@...hat.com>
Cc:	Xiaoguang Wang <wangxg.fnst@...fujitsu.com>,
	linux-ext4@...r.kernel.org, chrubis@...e.cz
Subject: Re: OpenPosix test case mmap_11-4 fails in ext4 filesystem

On Wed, May 21, 2014 at 02:55:12PM +0200, Lukáš Czerner wrote:
> > 
> > Recently I met the mmap_11-4 fails when running LTP in RHEL7.0RC. Attached is a
> > test program to reproduce this problem, which is written by Cyril.  Uncommenting the
> > msync() makes the test succeed in old linux distribution, such as RHEL6.5GA, but
> > fails in RHEL7.0RC.
> 
> please contact you Red Hat support to triage the issue within Red
> Hat.

This is a valid bug report which applies to upstream ext4.  It also
applies to tmpfs, by the way.

It would be good to get this test case (thank you very much for
writing it!) imported into xfstests, since the lack of this test is
why we didn't notice the bug when we added the fs/ext4/page_io.c code
paths.

BTW, there is one interesting thing about that changed when we moved
to page based I/O (years and years ago).  The requirement in mmap is
as Xiaoguang stated:

       A file is mapped in multiples of the page size.  For a file
       that is not a multi‐ ple of the page size, the remaining memory
       is zeroed when mapped, and writes to that region are not
       written out to the file.  The effect of changing the size of
       the underlying file of a mapping on the pages that correspond
       to added or removed regions of the file is unspecified.

Previously in Linux 2.2 (or was it 2.0; my memory is a bit fuzzy about
when we unified the buffer and page caches), when we copied the data
from the page cache to the buffer cache, we stopped at EOF.  This
implies that if the program modified the portion of the page mapped
beyond EOF, it would remain modified until the page was released and
the page was dropped from the page cache.

Now, when we call writepage() --- assuming the file system wasn't
buggy --- the bytes after EOF will get cleared.  If the user doesn't
call msync(), when this happens is unpredictable to the userspace
program, since it happens when writeback daemons decide to write back
the data.

See my modified test program to demonstrate this issue.  For ext3 and
xfs, you will see message:

nb: byte modified beyond EOF cleared after mysync

I think this is valid, or at least, not prohibited by the mmap()
interface specification, but it's not explicitly documented, and it
might be a little surprising.  Given that ext3 users have been seeing
this behavior for years and years, it's probably fine, so I'm
proposing that we change this; but we might want to make a note of it
in the man page.

Regards,

						- Ted

View attachment "mmap-test.c" of type "text/x-csrc" (2131 bytes)