linux-kernel - O_DIRECT and Btrfs == checksumming nightmare

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:	Fri, 08 Apr 2011 15:01:01 -0400
From:	Josef Bacik <josef@...hat.com>
To:	linux-fsdevel@...r.kernel.org
CC:	linux-btrfs <linux-btrfs@...r.kernel.org>,
	linux-kernel@...r.kernel.org, npiggin@...nel.dk, eparis@...hat.com,
	Chris Mason <chris.mason@...cle.com>
Subject: O_DIRECT and Btrfs == checksumming nightmare

Hello,

So I've been trying to track down checksumming errors Eric Paris was 
getting while running Windows 7 in qemu.  Turns out we had one valid 
problem (we don't deal well with reading with an iovec with two 
iov_base's that are the same), and we have a problem with the pages 
being changed in flight.  I'm not entirely sure the second thing is what 
is happening, but I'm looking at finding that out for sure soon.  But in 
the meantime I've crafted a fun little reproducer that will blow btrfs 
up quickly.  It just mmaps an anonymous range, fork()'s, and then one 
thread does writes/reads with the anonymous map and then the other one 
just sits there and loops and changes the anonymous map.  This will 
result in getting a -EIO on the reader thread pretty quickly and you get 
a bunch of checksum errors in your messages.

This is going to screw anybody who needs the pages to be stable during 
IO, and since its O_DIRECT we don't get to do any of our normal tricks 
to make sure things stay stable.  I even tried using set_memory_ro() to 
see if I could catch userspace modifying the page and it didn't do 
anything.  For now in btrfs the plan is to check the crc of the page 
when the IO completes (for writes) and if it's not create a bounce 
buffer and re-submit that.  This sucks, it would be good to have a way 
to make sure the pages were stable throughout the IO like we can with 
normal pages.  Nick, Chris said you had something in mind for this?  If 
you don't have time to do the actual work I can try and put together a 
fix if you can describe what to do.  I'm attaching my reproducer here in 
case anybody else wants to try it.  Thanks,

Josef

View attachment "modify-dio-in-flight.c" of type "text/plain" (2018 bytes)