netdev - Re: [PATCH V2 mlx5-next 14/14] vfio/mlx5: Use its own PCI reset

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20211020185721.GA334@nvidia.com>
Date:   Wed, 20 Oct 2021 15:57:21 -0300
From:   Jason Gunthorpe <jgg@...dia.com>
To:     Alex Williamson <alex.williamson@...hat.com>
Cc:     Yishai Hadas <yishaih@...dia.com>, bhelgaas@...gle.com,
        saeedm@...dia.com, linux-pci@...r.kernel.org, kvm@...r.kernel.org,
        netdev@...r.kernel.org, kuba@...nel.org, leonro@...dia.com,
        kwankhede@...dia.com, mgurtovoy@...dia.com, maorg@...dia.com
Subject: Re: [PATCH V2 mlx5-next 14/14] vfio/mlx5: Use its own PCI reset_done
 error handler

On Wed, Oct 20, 2021 at 11:45:14AM -0600, Alex Williamson wrote:
> On Wed, 20 Oct 2021 13:46:29 -0300
> Jason Gunthorpe <jgg@...dia.com> wrote:
> 
> > On Wed, Oct 20, 2021 at 11:46:07AM +0300, Yishai Hadas wrote:
> > 
> > > What is the expectation for a reasonable delay ? we may expect this system
> > > WQ to run only short tasks and be very responsive.  
> > 
> > If the expectation is that qemu will see the error return and the turn
> > around and issue FLR followed by another state operation then it does
> > seem strange that there would be a delay.
> > 
> > On the other hand, this doesn't seem that useful. If qemu tries to
> > migrate and the device fails then the migration operation is toast and
> > possibly the device is wrecked. It can't really issue a FLR without
> > coordinating with the VM, and it cannot resume the VM as the device is
> > now irrecoverably messed up.
> > 
> > If we look at this from a RAS perspective would would be useful here
> > is a way for qemu to request a fail safe migration data. This must
> > always be available and cannot fail.
> > 
> > When the failsafe is loaded into the device it would trigger the
> > device's built-in RAS features to co-ordinate with the VM driver and
> > recover. Perhaps qemu would also have to inject an AER or something.
> > 
> > Basically instead of the device starting in an "empty ready to use
> > state" it would start in a "failure detected, needs recovery" state.
> 
> The "fail-safe recovery state" is essentially the reset state of the
> device.

This is only the case if qemu does work to isolate the recently FLR'd
device from the VM until the VM acknowledges that it understands it is
FLR'd.

At least it would have to remove it from CPU access and the IOMMU, as
though the memory enable bit was cleared.

Is it reasonable to do this using just qemu, AER and no device
support?

> If a device enters an error state during migration, I would
> think the ultimate recovery procedure would be to abort the migration,
> send an AER to the VM, whereby the guest would trigger a reset, and
> the RAS capabilities of the guest would handle failing over to a
> multipath device, ejecting the failing device, etc.

Yes, this is my thinking, except I would not abort the migration but
continue on to the new hypervisor and then do the RAS recovery with
the new device.

Jason