Message-Id: <200707061137.46684.nigel@nigel.suspend2.net>
Date:	Fri, 6 Jul 2007 11:37:45 +1000
From:	Nigel Cunningham <nigel@...el.suspend2.net>
To:	Kyle Moffett <mrmacman_g4@....com>
Cc:	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	Pavel Machek <pavel@....cz>, "Rafael J. Wysocki" <rjw@...k.pl>,
	Matthew Garrett <mjg59@...f.ucam.org>,
	linux-kernel@...r.kernel.org, linux-pm@...ts.linux-foundation.org,
	Alan Stern <stern@...land.harvard.edu>
Subject: Re: [PATCH] Remove process freezer from suspend to RAM pathway

Hi.

On Friday 06 July 2007 11:19:32 Kyle Moffett wrote:
> On Jul 05, 2007, at 19:35:11, Nigel Cunningham wrote:
> > On Friday 06 July 2007 09:20:43 Benjamin Herrenschmidt wrote:
> >> No, the freezer creates all those places what are harmful for a  
> >> task to block because they will break the freezer :-)
> >
> > Nice try :) Okay then, you remove the freezer, try hibernating,  
> > then get back to me after you've fixed your filesystem because some  
> > process that wasn't frozen started writing things after the atomic  
> > copy (making the on disk filesystem inconsistent with the snapshot).
> 
> Umm, this thread is NOT ABOUT HIBERNATING!!!  Please go back and read  
> the subject, specifically the "suspend to RAM" parts :-D.  When your  
> hardware can put itself to sleep and atomically preserve memory as it  
> does so, you don't need an atomic copy.  For Real Suspend(TM) (IE:  
> Suspend-to-RAM), the list of things to do is short and simple:

We agreed a while back that you don't need the freezer for suspend to ram. As 
far as I was aware, we went off-topic, so the topic is out of date.

> 1)  Stop DMA and put most hardware into low-power states (stops all  
> interrupt sources)
> 2)  Ensure that the other CPUs have finished any trailing interrupt  
> handlers and put them to sleep
> 3)  Put the interrupt-controllers into low-power state
> 4)  Go to sleep
> 
> > As Pavel rightly said, you can get rid of the freezer, but you're  
> > only going to have to implement another one that does the  
> > essentially the same thing, even if it is at some other level.
> 
> How about a freezer whose job it is to "wait for pending hard  
> interrupts to complete when we have already guaranteed that we won't  
> get any more"?  That part should be really *REALLY* easy.  You don't  
> need to care about either userspace processes or kernel threads at  
> all.  Specifically, Step 1 consists of:
> 
> suspend_device(dev)
> {
> 	set_no_bind_flag(dev);
> 	for (dev->subdevices)
> 		suspend_device(dev);
> 	set_no_io_flag(dev);
> 	wait_for_in_progress_dma(dev);
> 	turn_off_interrupts(dev);
> 	go_to_low_power_state(dev);
> }
> 
> After you've set the "no_bind" flag, you won't get any *new*  
> subdevices trying to bind, therefore it's safe to iterate over the  
> list of present sub-devices and suspend them.  Once those are  
> suspended and in low-power states you can set a "no_io" flag to  
> prevent the driver from submitting more IO.  At that point you can  
> lazily wait for existing DMA/IO/interrupts to finish on the device,  
> since *NOBODY* will be submitting them anymore, and we certainly  
> aren't probing for new devices.  Then you can just turn off the power  
> to the device.  When all the leaf devices are off, the parent device  
> can be turned off because everything waiting on the leaf devices is  
> blocked on them and won't unblock until the parent device *AND* the  
> leaf device are turned on again, in that order.

For suspending, yes. For hibernating, that's not enough, because other 
processes can still be happily allocating and freeing memory, and will only 
get stopped when they try to do i/o or such like. If you're trying to make 
hibernation reliable, you need to be able to reliably check whether you're 
going to have enough storage for the image you're preparing, and enough 
memory for the atomic copy and so on. That's why the freezer is needed for 
hibernation. If you don't have it, any hibernation implementation you make is 
going to be only as reliable as the extent to which the system is otherwise 
idle.
 
> Scheduling and userspace are all still fully enabled in this  
> scenario.  Once all your devices are turned off, the only remaining  
> running threads will be those which haven't done IO since the  
> beginning of the suspend.  We can then disable preemption, turn off  
> the timer interrupts, and tell the other CPUs to park all their  
> remaining threads in schedule() and sleep.  Then we put the IRQ  
> controller to sleep and go to sleep ourselves.  If our driver model  
> locking is sufficient to handle putting a parent device to sleep  
> while threads are sleeping on a child device then there are exactly 0  
> problems.
> 
> Resuming is basically running the whole process in reverse.  Runtime- 
> suspend is achieved by not setting the 'no_io' or 'no_bind' flags and  
> putting selective device-subtrees to sleep without doing anything to  
> the rest of the system.

Fully agree when it comes to suspend to ram. There, this process should work, 
so far as I can see. But as I said, we went - to one degree or another - off 
topic, and did discuss hibernation too.

Regards,

Nigel
-- 
See http://www.tuxonice.net for Howtos, FAQs, mailing
lists, wiki and bugzilla info.

Content of type "application/pgp-signature" skipped

Powered by blists - more mailing lists