linux-kernel - Re: [RFC PATCH v1] cpu/suspend: Do a partial hotplug during suspend

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20241119092829.GF11903@noisy.programming.kicks-ass.net>
Date: Tue, 19 Nov 2024 10:28:29 +0100
From: Peter Zijlstra <peterz@...radead.org>
To: Saravana Kannan <saravanak@...gle.com>
Cc: Thomas Gleixner <tglx@...utronix.de>, kernel-team@...roid.com,
	linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH v1] cpu/suspend: Do a partial hotplug during suspend

On Mon, Nov 18, 2024 at 06:05:15PM -0800, Saravana Kannan wrote:
> The hotplug state machine goes through 100+ states when transitioning
> from online to offline. And on the way back, it goes through these
> states in reverse.
> 
> When a CPU goes offline, some of the states that occur after a CPU is
> powered off are about freeing up various per-CPU resources like
> kmalloc caches, pages, network buffers, etc. All of these states make
> sense when a CPU is permanently hotplugged off.
> 
> However, when offlining a CPU during suspend, we just want to power
> down the CPUs to that the system can enter suspend. In this scenario,
> we could simply stop the hotplug state machine right after the CPU has
> been power off. During resume, we can simply resume the CPU to an
> online state from the state where we paused the offline.
> 
> This save both time both during suspend and resume and it is
> proportional to the number of CPUs in the system. So, if systems with
> a large number of CPUs, we can expect this to have a huge amount of
> time saved.
> 
> On a Pixel 6, averaging across 100+ suspend/resumes cycles, the total
> time to power off 7 of the 8 CPUs goes from 51 ms down to 24 ms.
> Similarly, the average time to power off each individual CPU (they are
> different) also goes down by 50%.
> 
> The average time spent powering up CPUs goes down from 34 ms to 32 ms.
> Keep in mind that the time saved during resume is not easily
> quantified by looking at CPU onlining times. This is because the
> actual time savings comes later when per-CPU resources do not need to
> be reallocated and would speed up actions like allocations, etc that
> can pick up memory from per-CPU kmalloc caches, etc.
> 
> Signed-off-by: Saravana Kannan <saravanak@...gle.com>
> ---
> 
> Hi Thomas/Peter,
> 
> The hotplug state machine rewrite is great! Enables all kinds of
> optimizations for suspend/resume.
> 
> About this patch, I'm not sure if the exact state the hotplug state is
> paused at (CPUHP_WORKQUEUE_PREP) will work for all arch/boards, but
> this is the general idea.
> 
> If it works as is, great! At a glance, it looks like it should work
> though. None of the other stages between this and CPUHP_OFFLINE seem
> to be touching hardware.
> 
> If CPUHP_WORKQUEUE_PREP doesn't work, then we can make it a config
> option to select the state or an arch call or something along those
> lines.
> 
> What are your thoughts on this? How would you like me to proceed?

Well, if we push this one step further, why do we need hotplug at all?
Can't we just keep them up and idle?

That is, if we look at suspend_enter(), you'll note that
PM_SUSPEND_TO_IDLE happens before the whole disable_secondary_cpus()
thing.

So million-dollar question, can this pixel thing do suspend to idle?


Traditionally hybernate is the whole save-to-disk and power machine off
thing, and then there was suspend (to RAM) which was some dodgy as heck
BIOS thing (on x86) which required all non-boot CPUs to be 'dead'.

But does your (aaargh64) platform actually require us to take out the
non-boot CPUs, or is this just histerical raisins?