linux-kernel - Re: [BUG] 9p: data corruption with cache=mmap under concurrent stat/write

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-Id: <C48C493A-478A-4518-9EA2-CB52205CC1F0@barre.sh>
Date: Fri, 26 Dec 2025 14:13:40 +0100
From: Pierre Barre <pierre@...re.sh>
To: Christian Schoenebeck <linux_oss@...debyte.com>,
 asmadeus <asmadeus@...ewreck.org>
Cc: ericvh@...nel.org,
 lucho@...kov.net,
 v9fs@...ts.linux.dev,
 linux-kernel@...r.kernel.org,
 David Howells <dhowells@...hat.com>
Subject: Re: [BUG] 9p: data corruption with cache=mmap under concurrent
 stat/write

To clarify: the issue isn't with mmap access specifically. With cache=mmap (which enables CACHE_WRITEBACK), PostgreSQL uses regular write() calls that go through the page cache with writeback caching.

Best,
Pierre

> On 25 Dec 2025, at 15:52, Pierre Barre <pierre@...re.sh> wrote:
> 
> Hi Christian, Dominique,
> 
> Thank you for your reply and merry christmas.
> 
>>> unexpected data beyond EOF looks a lot like
>>> https://lkml.kernel.org/r/938162.1766233900@warthog.procyon.org.uk
>>> 
>>> could you try with this patch?
> 
> I will try this patch and report back.
> 
>> ... you are seeing a 2nd issue? getattr() output should not be related to 
>> mmap() access.
> 
> What's strange is that this issue doesn't occur during normal Postgres operation or while just running benchmarks. I initially encountered it while running du -hs during a pgbench benchmark, and I've since been able to reproduce it consistently with watch -n 0.1 tree -ah. Running the benchmarks for hours never trigger this bug, but it (almost) immediately occurs during du -hs / tree -ah.
> 
> Best,
> Pierre.
> 
> On Thu, Dec 25, 2025, at 11:23, Christian Schoenebeck wrote:
>> On Wednesday, 24 December 2025 23:33:58 CET Dominique Martinet wrote:
>>> Hi Pierre,
>>> 
>>> Pierre Barre wrote on Wed, Dec 24, 2025 at 03:29:01PM +0100:
>>>> I'm hitting data corruption using 9p with cache=mmap when stat() is called
>>>> concurrently with writes.
>>> Thanks for the report
>>> 
>>>> Environment:
>>>> - Kernel: v6.18.1-061801
>>>> - Mount options: cache=mmap
>>>> - Transport: unix
>>>> 
>>>> Reproducer:
>>>> 1. Mount 9p filesystem with cache=mmap
>>>> 2. Run PostgreSQL with data directory on 9p mount
>>>> 3. Run pgbench workload
>>>> 4. Simultaneously run `watch -n 0.1 tree -ah` on the data directory
>>>> 
>>>> PostgreSQL reports:
>>>>  ERROR: unexpected data beyond EOF in block N of relation "..."
>>> 
>>> unexpected data beyond EOF looks a lot like
>>> https://lkml.kernel.org/r/938162.1766233900@warthog.procyon.org.uk
>>> 
>>> could you try with this patch?
>> 
>> Pierre, I am also confident that this patch will fix the EOF data issue you 
>> encountered with PostgreSQL. However ...
>> 
>>>>  HINT: This has been seen to occur with buggy kernels
>>>> 
>>>> Analysis:
>>>> 
>>>> The issue appears to be race conditions in getattr/setattr when using
>>>> writeback caching:
>>>> 
>>>> 1. v9fs_vfs_getattr_dotl() condition checks `v9ses->cache` instead of
>>>> 
>>>>   `v9ses->cache & CACHE_WRITEBACK`, triggering writeback flush for
>>>>   any cache mode
>>>> 
>>>> 2. Both getattr and setattr call filemap_fdatawrite() which initiates
>>>> 
>>>>   writeback but doesn't wait for completion. The subsequent server
>>>>   stat/wstat sees stale file size.
>>>> 
>>>> Would using filemap_write_and_wait() instead be the correct fix?
>> 
>> ... you are seeing a 2nd issue? getattr() output should not be related to 
>> mmap() access.
>> 
>> /Christian