Message-ID: <20200221102130.GW4679@Air-de-Roger>
Date:   Fri, 21 Feb 2020 11:21:30 +0100
From:   Roger Pau Monné <roger.pau@...rix.com>
To:     "Durrant, Paul" <pdurrant@...zon.co.uk>
CC:     "Agarwal, Anchal" <anchalag@...zon.com>,
        "Valentin, Eduardo" <eduval@...zon.com>,
        "len.brown@...el.com" <len.brown@...el.com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "benh@...nel.crashing.org" <benh@...nel.crashing.org>,
        "x86@...nel.org" <x86@...nel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "pavel@....cz" <pavel@....cz>, "hpa@...or.com" <hpa@...or.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "sstabellini@...nel.org" <sstabellini@...nel.org>,
        "fllinden@...ozn.com" <fllinden@...ozn.com>,
        "Kamata, Munehisa" <kamatam@...zon.com>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "xen-devel@...ts.xenproject.org" <xen-devel@...ts.xenproject.org>,
        "Singh, Balbir" <sblbir@...zon.com>,
        "axboe@...nel.dk" <axboe@...nel.dk>,
        "konrad.wilk@...cle.com" <konrad.wilk@...cle.com>,
        "bp@...en8.de" <bp@...en8.de>,
        "boris.ostrovsky@...cle.com" <boris.ostrovsky@...cle.com>,
        "jgross@...e.com" <jgross@...e.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>,
        "rjw@...ysocki.net" <rjw@...ysocki.net>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "vkuznets@...hat.com" <vkuznets@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "Woodhouse, David" <dwmw@...zon.co.uk>
Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for
 PM suspend and hibernation

On Fri, Feb 21, 2020 at 09:56:54AM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monné <roger.pau@...rix.com>
> > Sent: 21 February 2020 09:22
> > To: Durrant, Paul <pdurrant@...zon.co.uk>
> > Cc: Agarwal, Anchal <anchalag@...zon.com>; Valentin, Eduardo
> > <eduval@...zon.com>; len.brown@...el.com; peterz@...radead.org;
> > benh@...nel.crashing.org; x86@...nel.org; linux-mm@...ck.org;
> > pavel@....cz; hpa@...or.com; tglx@...utronix.de; sstabellini@...nel.org;
> > fllinden@...ozn.com; Kamata, Munehisa <kamatam@...zon.com>;
> > mingo@...hat.com; xen-devel@...ts.xenproject.org; Singh, Balbir
> > <sblbir@...zon.com>; axboe@...nel.dk; konrad.wilk@...cle.com;
> > bp@...en8.de; boris.ostrovsky@...cle.com; jgross@...e.com;
> > netdev@...r.kernel.org; linux-pm@...r.kernel.org; rjw@...ysocki.net;
> > linux-kernel@...r.kernel.org; vkuznets@...hat.com; davem@...emloft.net;
> > Woodhouse, David <dwmw@...zon.co.uk>
> > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks
> > for PM suspend and hibernation
> > 
> > On Thu, Feb 20, 2020 at 05:01:52PM +0000, Durrant, Paul wrote:
> > > > > Hopefully what I said above illustrates why it may not be 100%
> > common.
> > > >
> > > > Yes, that's fine. I don't expect it to be 100% common (as I guess
> > > > that the hooks will have different prototypes), but I expect
> > > > that routines can be shared, and that the approach taken can be the
> > > > same.
> > > >
> > > > For example one necessary difference will be that xenbus initiated
> > > > suspend won't close the PV connection, in case suspension fails. On PM
> > > > suspend you seem to always close the connection beforehand, so you
> > > > will always have to re-negotiate on resume even if suspension failed.
> > > >
> > > > What I'm mostly worried about is the different approach to ring
> > > > draining. Ie: either xenbus is changed to freeze the queues and drain
> > > > the shared rings, or PM uses the already existing logic of not
> > > > flushing the rings an re-issuing in-flight requests on resume.
> > > >
> > >
> > > Yes, that's needs consideration. I don’t think the same semantic can be
> > suitable for both. E.g. in a xen-suspend we need to freeze with as little
> > processing as possible to avoid dirtying RAM late in the migration cycle,
> > and we know that in-flight data can wait. But in a transition to S4 we
> > need to make sure that at least all the in-flight blkif requests get
> > completed, since they probably contain bits of the guest's memory image
> > and that's not going to get saved any other way.
> > 
> > Thanks, that makes sense and something along this lines should be
> > added to the commit message IMO.
> > 
> > Wondering about S4, shouldn't we expect the queues to already be
> > empty? As any subsystem that wanted to store something to disk should
> > make sure requests have been successfully completed before
> > suspending.
> 
> What about writing the suspend image itself? Normal filesystem I/O
> will have been flushed of course, but whatever vestigial kernel
> actually writes out the hibernation file may well expect a final
> D0->D3 on the storage device to cause a flush.

Hm, I have no idea really. I think whatever writes to the disk before
suspend should actually make sure requests have completed, but what
you suggest might also be a possibility.

Can you figure out whether there are requests on the ring or in the
queue before suspending?

> Again, I don't know the specifics for Linux (and Windows actually
> uses an incarnation of the crash kernel to do the job, which brings
> with it a whole other set of complexity as far as PV drivers go).

That seems extremely complex, I'm sure there's a reason for it :).

Thanks, Roger.

Powered by blists - more mailing lists