linux-kernel - RE: [PATCH] cpu/hotplug: Allow the CPU in CPU_UP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Tue, 23 Nov 2021 23:50:37 +0000
From:   "Longpeng (Mike, Cloud Infrastructure Service Product Dept.)" 
        <longpeng2@...wei.com>
To:     Dongli Zhang <dongli.zhang@...cle.com>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>
CC:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Gonglei (Arei)" <arei.gonglei@...wei.com>,
        "x86@...nel.org" <x86@...nel.org>,
        "xen-devel@...ts.xenproject.org" <xen-devel@...ts.xenproject.org>,
        "Peter Zijlstra" <peterz@...radead.org>,
        Ingo Molnar <mingo@...nel.org>,
        "Valentin Schneider" <valentin.schneider@....com>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Juergen Gross <jgross@...e.com>,
        "Stefano Stabellini" <sstabellini@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        Dave Hansen <dave.hansen@...ux.intel.com>,
        "H. Peter Anvin" <hpa@...or.com>
Subject: RE: [PATCH] cpu/hotplug: Allow the CPU in CPU_UP_PREPARE state to be
 brought up again.



> -----Original Message-----
> From: Dongli Zhang [mailto:dongli.zhang@...cle.com]
> Sent: Wednesday, November 24, 2021 5:22 AM
> To: Sebastian Andrzej Siewior <bigeasy@...utronix.de>; Longpeng (Mike, Cloud
> Infrastructure Service Product Dept.) <longpeng2@...wei.com>
> Cc: linux-kernel@...r.kernel.org; Gonglei (Arei) <arei.gonglei@...wei.com>;
> x86@...nel.org; xen-devel@...ts.xenproject.org; Peter Zijlstra
> <peterz@...radead.org>; Ingo Molnar <mingo@...nel.org>; Valentin Schneider
> <valentin.schneider@....com>; Boris Ostrovsky <boris.ostrovsky@...cle.com>;
> Juergen Gross <jgross@...e.com>; Stefano Stabellini <sstabellini@...nel.org>;
> Thomas Gleixner <tglx@...utronix.de>; Ingo Molnar <mingo@...hat.com>; Borislav
> Petkov <bp@...en8.de>; Dave Hansen <dave.hansen@...ux.intel.com>; H. Peter
> Anvin <hpa@...or.com>
> Subject: Re: [PATCH] cpu/hotplug: Allow the CPU in CPU_UP_PREPARE state to be
> brought up again.
> 
> Tested-by: Dongli Zhang <dongli.zhang@...cle.com>
> 
> 
> The bug fixed by commit 53fafdbb8b21 ("KVM: x86: switch KVMCLOCK base to
> monotonic raw clock") may leave the cpu_hotplug_state at CPU_UP_PREPARE. As a
> result, to online this CPU again (even after removal) is always failed.
> 
> I have tested that this patch works well to workaround the issue, by introducing
> either a mdeley(11000) or while(1); to start_secondary(). That is, to online
> the
> same CPU again is successful even after initial do_boot_cpu() failure.
> 
> 1. add mdelay(11000) or while(1); to the start_secondary().
> 
> 2. to online CPU is failed at do_boot_cpu().
> 

Thanks for your testing :)

Does the cpu4 spin in wait_for_master_cpu() in your case ?

> 3. to online CPU again is failed without this patch.
> 
> # echo 1 > /sys/devices/system/cpu/cpu4/online
> -su: echo: write error: Input/output error
> 
> 4. to online CPU again is successful with this patch.
> 
> Thank you very much!
> 
> Dongli Zhang
> 
> On 11/22/21 7:47 AM, Sebastian Andrzej Siewior wrote:
> > From: "Longpeng(Mike)" <longpeng2@...wei.com>
> >
> > A CPU will not show up in virtualized environment which includes an
> > Enclave. The VM splits its resources into a primary VM and a Enclave
> > VM. While the Enclave is active, the hypervisor will ignore all requests
> > to bring up a CPU and this CPU will remain in CPU_UP_PREPARE state.
> > The kernel will wait up to ten seconds for CPU to show up
> > (do_boot_cpu()) and then rollback the hotplug state back to
> > CPUHP_OFFLINE leaving the CPU state in CPU_UP_PREPARE. The CPU state is
> > set back to CPUHP_TEARDOWN_CPU during the CPU_POST_DEAD stage.
> >
> > After the Enclave VM terminates, the primary VM can bring up the CPU
> > again.
> >
> > Allow to bring up the CPU if it is in the CPU_UP_PREPARE state.
> >
> > [bigeasy: Rewrite commit description.]
> >
> > Signed-off-by: Longpeng(Mike) <longpeng2@...wei.com>
> > Signed-off-by: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
> > Link:
> https://urldefense.com/v3/__https://lore.kernel.org/r/20210901051143.2752-1
> -longpeng2@...wei.com__;!!ACWV5N9M2RV99hQ!d4sCCXMQV7ekFwpd21vo1_9K-m5h4VZ-g
> E8Z62PLL58DT4VJ6StH57TR_KpBdbwhBE0$
> > ---
> >
> > For XEN: this changes the behaviour as it allows to invoke
> > cpu_initialize_context() again should it have have earlier. I *think*
> > this is okay and would to bring up the CPU again should the memory
> > allocation in cpu_initialize_context() fail.
> >
> >  kernel/smpboot.c | 7 +++++++
> >  1 file changed, 7 insertions(+)
> >
> > diff --git a/kernel/smpboot.c b/kernel/smpboot.c
> > index f6bc0bc8a2aab..34958d7fe2c1c 100644
> > --- a/kernel/smpboot.c
> > +++ b/kernel/smpboot.c
> > @@ -392,6 +392,13 @@ int cpu_check_up_prepare(int cpu)
> >  		 */
> >  		return -EAGAIN;
> >
> > +	case CPU_UP_PREPARE:
> > +		/*
> > +		 * Timeout while waiting for the CPU to show up. Allow to try
> > +		 * again later.
> > +		 */
> > +		return 0;
> > +
> >  	default:
> >
> >  		/* Should not happen.  Famous last words. */
> >