linux-kernel - Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for PM suspend and hibernation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Fri, 21 Feb 2020 12:51:03 +0100
From:   Roger Pau Monné <roger.pau@...rix.com>
To:     "Durrant, Paul" <pdurrant@...zon.co.uk>
CC:     "Agarwal, Anchal" <anchalag@...zon.com>,
        "Valentin, Eduardo" <eduval@...zon.com>,
        "len.brown@...el.com" <len.brown@...el.com>,
        "peterz@...radead.org" <peterz@...radead.org>,
        "benh@...nel.crashing.org" <benh@...nel.crashing.org>,
        "x86@...nel.org" <x86@...nel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "pavel@....cz" <pavel@....cz>, "hpa@...or.com" <hpa@...or.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "sstabellini@...nel.org" <sstabellini@...nel.org>,
        "fllinden@...ozn.com" <fllinden@...ozn.com>,
        "Kamata, Munehisa" <kamatam@...zon.com>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "xen-devel@...ts.xenproject.org" <xen-devel@...ts.xenproject.org>,
        "Singh, Balbir" <sblbir@...zon.com>,
        "axboe@...nel.dk" <axboe@...nel.dk>,
        "konrad.wilk@...cle.com" <konrad.wilk@...cle.com>,
        "bp@...en8.de" <bp@...en8.de>,
        "boris.ostrovsky@...cle.com" <boris.ostrovsky@...cle.com>,
        "jgross@...e.com" <jgross@...e.com>,
        "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
        "linux-pm@...r.kernel.org" <linux-pm@...r.kernel.org>,
        "rjw@...ysocki.net" <rjw@...ysocki.net>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "vkuznets@...hat.com" <vkuznets@...hat.com>,
        "davem@...emloft.net" <davem@...emloft.net>,
        "Woodhouse, David" <dwmw@...zon.co.uk>
Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks for
 PM suspend and hibernation

On Fri, Feb 21, 2020 at 10:33:42AM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monné <roger.pau@...rix.com>
> > Sent: 21 February 2020 10:22
> > To: Durrant, Paul <pdurrant@...zon.co.uk>
> > Cc: Agarwal, Anchal <anchalag@...zon.com>; Valentin, Eduardo
> > <eduval@...zon.com>; len.brown@...el.com; peterz@...radead.org;
> > benh@...nel.crashing.org; x86@...nel.org; linux-mm@...ck.org;
> > pavel@....cz; hpa@...or.com; tglx@...utronix.de; sstabellini@...nel.org;
> > fllinden@...ozn.com; Kamata, Munehisa <kamatam@...zon.com>;
> > mingo@...hat.com; xen-devel@...ts.xenproject.org; Singh, Balbir
> > <sblbir@...zon.com>; axboe@...nel.dk; konrad.wilk@...cle.com;
> > bp@...en8.de; boris.ostrovsky@...cle.com; jgross@...e.com;
> > netdev@...r.kernel.org; linux-pm@...r.kernel.org; rjw@...ysocki.net;
> > linux-kernel@...r.kernel.org; vkuznets@...hat.com; davem@...emloft.net;
> > Woodhouse, David <dwmw@...zon.co.uk>
> > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add callbacks
> > for PM suspend and hibernation
> > 
> > On Fri, Feb 21, 2020 at 09:56:54AM +0000, Durrant, Paul wrote:
> > > > -----Original Message-----
> > > > From: Roger Pau Monné <roger.pau@...rix.com>
> > > > Sent: 21 February 2020 09:22
> > > > To: Durrant, Paul <pdurrant@...zon.co.uk>
> > > > Cc: Agarwal, Anchal <anchalag@...zon.com>; Valentin, Eduardo
> > > > <eduval@...zon.com>; len.brown@...el.com; peterz@...radead.org;
> > > > benh@...nel.crashing.org; x86@...nel.org; linux-mm@...ck.org;
> > > > pavel@....cz; hpa@...or.com; tglx@...utronix.de;
> > sstabellini@...nel.org;
> > > > fllinden@...ozn.com; Kamata, Munehisa <kamatam@...zon.com>;
> > > > mingo@...hat.com; xen-devel@...ts.xenproject.org; Singh, Balbir
> > > > <sblbir@...zon.com>; axboe@...nel.dk; konrad.wilk@...cle.com;
> > > > bp@...en8.de; boris.ostrovsky@...cle.com; jgross@...e.com;
> > > > netdev@...r.kernel.org; linux-pm@...r.kernel.org; rjw@...ysocki.net;
> > > > linux-kernel@...r.kernel.org; vkuznets@...hat.com;
> > davem@...emloft.net;
> > > > Woodhouse, David <dwmw@...zon.co.uk>
> > > > Subject: Re: [Xen-devel] [RFC PATCH v3 06/12] xen-blkfront: add
> > callbacks
> > > > for PM suspend and hibernation
> > > >
> > > > On Thu, Feb 20, 2020 at 05:01:52PM +0000, Durrant, Paul wrote:
> > > > > > > Hopefully what I said above illustrates why it may not be 100%
> > > > common.
> > > > > >
> > > > > > Yes, that's fine. I don't expect it to be 100% common (as I guess
> > > > > > that the hooks will have different prototypes), but I expect
> > > > > > that routines can be shared, and that the approach taken can be
> > the
> > > > > > same.
> > > > > >
> > > > > > For example one necessary difference will be that xenbus initiated
> > > > > > suspend won't close the PV connection, in case suspension fails.
> > On PM
> > > > > > suspend you seem to always close the connection beforehand, so you
> > > > > > will always have to re-negotiate on resume even if suspension
> > failed.
> > > > > >
> > > > > > What I'm mostly worried about is the different approach to ring
> > > > > > draining. Ie: either xenbus is changed to freeze the queues and
> > drain
> > > > > > the shared rings, or PM uses the already existing logic of not
> > > > > > flushing the rings an re-issuing in-flight requests on resume.
> > > > > >
> > > > >
> > > > > Yes, that's needs consideration. I don’t think the same semantic can
> > be
> > > > suitable for both. E.g. in a xen-suspend we need to freeze with as
> > little
> > > > processing as possible to avoid dirtying RAM late in the migration
> > cycle,
> > > > and we know that in-flight data can wait. But in a transition to S4 we
> > > > need to make sure that at least all the in-flight blkif requests get
> > > > completed, since they probably contain bits of the guest's memory
> > image
> > > > and that's not going to get saved any other way.
> > > >
> > > > Thanks, that makes sense and something along this lines should be
> > > > added to the commit message IMO.
> > > >
> > > > Wondering about S4, shouldn't we expect the queues to already be
> > > > empty? As any subsystem that wanted to store something to disk should
> > > > make sure requests have been successfully completed before
> > > > suspending.
> > >
> > > What about writing the suspend image itself? Normal filesystem I/O
> > > will have been flushed of course, but whatever vestigial kernel
> > > actually writes out the hibernation file may well expect a final
> > > D0->D3 on the storage device to cause a flush.
> > 
> > Hm, I have no idea really. I think whatever writes to the disk before
> > suspend should actually make sure requests have completed, but what
> > you suggest might also be a possibility.
> > 
> > Can you figure out whether there are requests on the ring or in the
> > queue before suspending?
> 
> Well there's clearly pending stuff in the ring if rsp_prod != req_prod :-)

Right, I assume there's no document that states what's the expected
state for queues &c when switching PM states, so we have to assume
that there might be in-flight requests on the ring and in the driver
queues.

> As for internal queues, I don't know how blkfront manages that (or whether it has any pending work queue at all).

There are no internal queues, just the generic ones from blk_mq which
every block device has IIRC.

Thanks, Roger.