[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4DB1C27A-1B89-468A-9103-80DEDBF1A091@oracle.com>
Date: Thu, 17 Aug 2023 15:57:39 +0000
From: Chuck Lever III <chuck.lever@...cle.com>
To: Maxim Levitsky <mlevitsk@...hat.com>
CC: Linux NFS Mailing List <linux-nfs@...r.kernel.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"kvm@...r.kernel.org" <kvm@...r.kernel.org>
Subject: Re: Commit 'sunrpc: Use sendmsg(MSG_SPLICE_PAGES) rather then
sendpage' broke O_DIRECT over NFS
> On Aug 17, 2023, at 11:52 AM, Maxim Levitsky <mlevitsk@...hat.com> wrote:
>
> Hi!
>
> I just updated my developement systems to 6.5-rc6 (from 6.4) and now I can't start a VM
> with a disk which is mounted over the NFS.
>
> The VM has two qcow2 files, one depends on another and qemu opens both.
>
> This is the command line of qemu:
>
> -drive if=none,id=os_image,file=./disk_s1.qcow2,aio=native,discard=unmap,cache=none
>
> The disk_s1.qcow2 depends on disk_s0.qcow2
>
> However this is what I get:
>
> qemu-system-x86_64: -drive if=none,id=os_image,file=./disk_s1.qcow2,aio=native,discard=unmap,cache=none: Could not open backing file: Could not open './QFI?': No such file or directory
>
> 'QFI?' is qcow2 file signature, which signals that there might be some nasty corruption happening.
>
> The program was supposed to read a field inside the disk_s1.qcow2 file which should read 'disk_s0.qcow2'
> but instead it seems to read the first 4 bytes of the file.
>
>
> Bisect leads to the above commit. Reverting it was not possible due to many changes.
>
> Both the client and the server were tested with the 6.5-rc6 kernel, but once rebooting the server into
> the 6.4, the bug disappeared, thus I did a bisect on the server.
>
> When I tested a version before the offending commit on the server, the 6.5-rc6 client was able to work with it,
> which increases the chances that the bug is in nfsd.
>
> Switching qemu to use write back paging also helps (aio=threads,discard=unmap,cache=writeback)
> The client and the server (both 6.5-rc6) work with this configuration.
>
> Running the VM on the same machine (also 6.5-rc6) where the VM disk is located (thus avoiding NFS) works as well.
>
> I tested several VMs that I have, all are affected in the same way.
>
> I run somewhat outdated qemu, but running the latest qemu doesn't make a difference.
>
> I use nfs4.
>
> I can test patches and provide more info if needed.
Linus just merged a possible fix for this issue. See:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/ master
--
Chuck Lever
Powered by blists - more mailing lists