linux-kernel - Re: [PATCH] powerpc: on crash, kexec'ed kernel needs all CPUs are online

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20151016132943.1386fda6@voom.fritz.box>
Date:	Fri, 16 Oct 2015 13:29:43 +1100
From:	David Gibson <dgibson@...hat.com>
To:	Laurent Vivier <lvivier@...hat.com>
Cc:	Benjamin Herrenschmidt <benh@...nel.crashing.org>,
	Paul Mackerras <paulus@...ba.org>,
	Michael Ellerman <mpe@...erman.id.au>,
	linuxppc-dev@...ts.ozlabs.org, linux-kernel@...r.kernel.org,
	thuth@...hat.com
Subject: Re: [PATCH] powerpc: on crash, kexec'ed kernel needs all CPUs are
 online

On Thu, 15 Oct 2015 21:00:58 +0200
Laurent Vivier <lvivier@...hat.com> wrote:

> On kexec, all secondary offline CPUs are onlined before
> starting the new kernel, this is not done in the case of kdump.
> 
> If kdump is configured and a kernel crash occurs whereas
> some secondaries CPUs are offline (SMT=off),
> the new kernel is not able to start them and displays some
> "Processor X is stuck.".
> 
> Starting with POWER8, subcore logic relies on all threads of
> core being booted. So, on startup kernel tries to start all
> threads, and asks OPAL (or RTAS) to start all CPUs (including
> threads). If a CPU has been offlined by the previous kernel,
> it has not been returned to OPAL, and thus OPAL cannot restart
> it: this CPU has been lost...
> 
> Signed-off-by: Laurent Vivier <lvivier@...hat.com>

Nice analysis of the problem.  But, I'm a bit uneasy about this approach
to fixing it: Onlining potentially hundreds of CPU threads seems like
a risky operation in a kernel that's already crashed.

I don't have a terribly clear idea of what is the best way to address
this.  Here's a few ideas in the right general direction:

  * I'm already looking into a kdump userspace fixes to stop it
    attempting to bring up secondary CPUs

  * A working kernel option to say "only allow this many online cpus
    ever" which we could pass to the kdump kernel would be nice

  * Paulus had an idea about offline threads returning themselves
    directly to OPAL by kicking a flag at kdump/kexec time.

BenH, Paulus,

OPAL <-> kernel cpu transitions don't seem to work quite how I thought
they would.  IIUC there's a register we can use to directly control
which threads on a core are active.  Given that I would have thought
cpu "ownership" OPAL vs. kernel would be on a per-core, rather than
per-thread basis.

Is there some way we can change the CPU onlining / offlining code so
that if threads aren't in OPAL, we directly enable them, rather than
just hoping they're in a nap loop somewhere?

-- 
David Gibson <dgibson@...hat.com>
Senior Software Engineer, Virtualization, Red Hat

Content of type "application/pgp-signature" skipped