linux-kernel - Re: [PATCH v2 1/2] KVM: Use syscore_ops instead of reboot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87wmtk9u46.fsf@email.froward.int.ebiederm.org>
Date:   Mon, 11 Dec 2023 17:50:33 -0600
From:   "Eric W. Biederman" <ebiederm@...ssion.com>
To:     "Gowans, James" <jgowans@...zon.com>
Cc:     "Graf (AWS), Alexander" <graf@...zon.de>,
        "seanjc@...gle.com" <seanjc@...gle.com>,
        Schönherr, Jan H. <jschoenh@...zon.de>,
        "yuzenghui@...wei.com" <yuzenghui@...wei.com>,
        "kvm-riscv@...ts.infradead.org" <kvm-riscv@...ts.infradead.org>,
        "kexec@...ts.infradead.org" <kexec@...ts.infradead.org>,
        "james.morse@....com" <james.morse@....com>,
        "oliver.upton@...ux.dev" <oliver.upton@...ux.dev>,
        "suzuki.poulose@....com" <suzuki.poulose@....com>,
        "chenhuacai@...nel.org" <chenhuacai@...nel.org>,
        "atishp@...shpatra.org" <atishp@...shpatra.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "maz@...nel.org" <maz@...nel.org>,
        "pbonzini@...hat.com" <pbonzini@...hat.com>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "kvmarm@...ts.linux.dev" <kvmarm@...ts.linux.dev>,
        "anup@...infault.org" <anup@...infault.org>,
        "aleksandar.qemu.devel@...il.com" <aleksandar.qemu.devel@...il.com>
Subject: Re: [PATCH v2 1/2] KVM: Use syscore_ops instead of reboot_notifier
 to hook restart/shutdown

"Gowans, James" <jgowans@...zon.com> writes:

> On Mon, 2023-12-11 at 09:54 +0200, James Gowans wrote:
>> > 
>> > What problem are you running into with your rebase that worked with
>> > reboot notifiers that is not working with syscore_shutdown?
>> 
>> Prior to this commit [1] which changed KVM from reboot notifiers to
>> syscore_ops, KVM's reboot notifier shutdown callback was invoked on
>> kexec via kernel_restart_prepare.
>> 
>> After this commit, KVM is not being shut down because currently the
>> kexec flow does not call syscore_shutdown.
>
> I think I missed what you're asking here; you're asking for a reproducer
> for the specific failure? 
>
> 1. Launch a QEMU VM with -enable-kvm flag
>
> 2. Do an immediate (-f flag) kexec:
> kexec -f --reuse-cmdline ./bzImage 
>
> Somewhere after doing the RET to new kernel in the relocate_kernel asm
> function the new kernel starts triple faulting; I can't exactly figure
> out where but I think it has to do with the new kernel trying to modify
> CR3 while the VMXE bit is still set in CR4 causing the triple fault.
>
> If KVM has been shut down via the shutdown callback, or alternatively if
> the QEMU process has actually been killed first (by not doing a -f exec)
> then the VMXE bit is clear and the kexec goes smoothly.
>
> So, TL;DR: kexec -f use to work with a KVM VM active, now it goes into a
> triple fault crash.

You mentioned I rebase so I thought your were backporting kernel patches.
By rebase do you mean you porting your userspace to a newer kernel?


In any event I believe the bug with respect to kexec was introduced in
commit 6f389a8f1dd2 ("PM / reboot: call syscore_shutdown() after
disable_nonboot_cpus()").  That is where syscore_shutdown was removed
from kernel_restart_prepare().

At this point it looks like someone just needs to add the missing
syscore_shutdown call into kernel_kexec() right after
migrate_to_reboot_cpu() is called.

That said I am not seeing the reboot notifiers being called on the kexec
path either so your issue with kvm might be deeper.

Eric