| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <0640effb-5c65-5f52-a58f-4a12104b189b@loongson.cn>
Date: Mon, 6 May 2024 14:32:00 +0800
From: maobibo <maobibo@...ngson.cn>
To: Huacai Chen <chenhuacai@...nel.org>
Cc: Tianrui Zhao <zhaotianrui@...ngson.cn>, Juergen Gross <jgross@...e.com>,
Paolo Bonzini <pbonzini@...hat.com>, Jonathan Corbet <corbet@....net>,
loongarch@...ts.linux.dev, linux-kernel@...r.kernel.org,
virtualization@...ts.linux.dev, kvm@...r.kernel.org
Subject: Re: [PATCH v8 0/6] LoongArch: Add pv ipi support on LoongArch VM
On 2024/5/6 上午9:45, Huacai Chen wrote:
> Hi, Bibo,
>
> I have done an off-list discussion with some KVM experts, and they
> think user-space have its right to know PV features, so cpucfg
> solution is acceptable.
>
> And I applied this series with some modifications at
> https://git.kernel.org/pub/scm/linux/kernel/git/chenhuacai/linux-loongson.git/log/?h=loongarch-kvm
> You can test it now. But it seems the upstream qemu cannot enable PV IPI now.
VM with 128/256 vcpus boots with this series in loongarch-kvm branch.
And pv ipi works by information "cat /proc/interrupts". There need small
modification with qemu like this, and we
will submit the patch to qemu after it is merged.
diff --git a/hw/loongarch/virt.c b/hw/loongarch/virt.c
index 441d764843..9f7556cd93 100644
--- a/hw/loongarch/virt.c
+++ b/hw/loongarch/virt.c
@@ -15,6 +15,8 @@
#include "sysemu/runstate.h"
#include "sysemu/reset.h"
#include "sysemu/rtc.h"
+#include "sysemu/tcg.h"
+#include "sysemu/kvm.h"
#include "hw/loongarch/virt.h"
#include "exec/address-spaces.h"
#include "hw/irq.h"
@@ -786,12 +788,18 @@ static void loongarch_qemu_write(void *opaque,
hwaddr addr,
static uint64_t loongarch_qemu_read(void *opaque, hwaddr addr,
unsigned size)
{
+ uint64_t ret = 0;
+
switch (addr) {
case VERSION_REG:
return 0x11ULL;
case FEATURE_REG:
- return 1ULL << IOCSRF_MSI | 1ULL << IOCSRF_EXTIOI |
+ ret =1ULL << IOCSRF_MSI | 1ULL << IOCSRF_EXTIOI |
1ULL << IOCSRF_CSRIPI;
+ if (kvm_enabled()) {
+ ret |= 1ULL << IOCSRF_VM;
+ }
+ return ret;
case VENDOR_REG:
return 0x6e6f73676e6f6f4cULL; /* "Loongson" */
case CPUNAME_REG:
Regards
Bibo Mao
>
> I will reply to other patches about my modifications.
>
> Huacai
>
> On Sun, Apr 28, 2024 at 6:05 PM Bibo Mao <maobibo@...ngson.cn> wrote:
>>
>> On physical machine, ipi HW uses IOCSR registers, however there is trap
>> into hypervisor when vcpu accesses IOCSR registers if system is in VM
>> mode. SWI is a interrupt mechanism like SGI on ARM, software can send
>> interrupt to CPU, only that on LoongArch SWI can only be sent to local CPU
>> now. So SWI can not used for IPI on real HW system, however it can be used
>> on VM when combined with hypercall method. IPI can be sent with hypercall
>> method and SWI interrupt is injected to vcpu, vcpu can treat SWI
>> interrupt as IPI.
>>
>> With PV IPI supported, there is one trap with IPI sending, however with IPI
>> receiving there is no trap. with IOCSR HW ipi method, there will be one
>> trap with IPI sending and two trap with ipi receiving.
>>
>> Also IPI multicast support is added for VM, the idea comes from x86 PV ipi.
>> IPI can be sent to 128 vcpus in one time. With IPI multicast support, trap
>> will be reduced greatly.
>>
>> Here is the microbenchmarck data with "perf bench futex wake" testcase on
>> 3C5000 single-way machine, there are 16 cpus on 3C5000 single-way machine,
>> VM has 16 vcpus also. The benchmark data is ms time unit to wakeup 16
>> threads, the performance is better if data is smaller.
>>
>> physical machine 0.0176 ms
>> VM original 0.1140 ms
>> VM with pv ipi patch 0.0481 ms
>>
>> It passes to boot with 128/256 vcpus, and passes to run runltp command
>> with package ltp-20230516.
>>
>> ---
>> v7 --- v8:
>> 1. Remove kernel PLV mode checking with cpucfg emulation for hypervisor
>> feature inquiry.
>> 2. Remove document about loongarch hypercall ABI per request of huacai,
>> will add English/Chinese doc at the same time in later.
>>
>> v6 --- v7:
>> 1. Refine LoongArch virt document by review comments.
>> 2. Add function kvm_read_reg()/kvm_write_reg() in hypercall emulation,
>> and later it can be used for other trap emulations.
>>
>> v5 --- v6:
>> 1. Add privilege checking when emulating cpucfg at index 0x4000000 --
>> 0x400000FF, return 0 if not executed at kernel mode.
>> 2. Add document about LoongArch pv ipi with new creatly directory
>> Documentation/virt/kvm/loongarch/
>> 3. Fix pv ipi handling in kvm backend function kvm_pv_send_ipi(),
>> where min should plus BITS_PER_LONG with second bitmap, otherwise
>> VM with more than 64 vpus fails to boot.
>> 4. Adjust patch order and code refine with review comments.
>>
>> v4 --- v5:
>> 1. Refresh function/macro name from review comments.
>>
>> v3 --- v4:
>> 1. Modfiy pv ipi hook function name call_func_ipi() and
>> call_func_single_ipi() with send_ipi_mask()/send_ipi_single(), since pv
>> ipi is used for both remote function call and reschedule notification.
>> 2. Refresh changelog.
>>
>> v2 --- v3:
>> 1. Add 128 vcpu ipi multicast support like x86
>> 2. Change cpucfg base address from 0x10000000 to 0x40000000, in order
>> to avoid confliction with future hw usage
>> 3. Adjust patch order in this patchset, move patch
>> Refine-ipi-ops-on-LoongArch-platform to the first one.
>>
>> v1 --- v2:
>> 1. Add hw cpuid map support since ipi routing uses hw cpuid
>> 2. Refine changelog description
>> 3. Add hypercall statistic support for vcpu
>> 4. Set percpu pv ipi message buffer aligned with cacheline
>> 5. Refine pv ipi send logic, do not send ipi message with if there is
>> pending ipi message.
>> ---
>> Bibo Mao (6):
>> LoongArch/smp: Refine some ipi functions on LoongArch platform
>> LoongArch: KVM: Add hypercall instruction emulation support
>> LoongArch: KVM: Add cpucfg area for kvm hypervisor
>> LoongArch: KVM: Add vcpu search support from physical cpuid
>> LoongArch: KVM: Add pv ipi support on kvm side
>> LoongArch: Add pv ipi support on guest kernel side
>>
>> arch/loongarch/Kconfig | 9 +
>> arch/loongarch/include/asm/Kbuild | 1 -
>> arch/loongarch/include/asm/hardirq.h | 5 +
>> arch/loongarch/include/asm/inst.h | 1 +
>> arch/loongarch/include/asm/irq.h | 10 +-
>> arch/loongarch/include/asm/kvm_host.h | 27 +++
>> arch/loongarch/include/asm/kvm_para.h | 155 ++++++++++++++++++
>> arch/loongarch/include/asm/kvm_vcpu.h | 11 ++
>> arch/loongarch/include/asm/loongarch.h | 11 ++
>> arch/loongarch/include/asm/paravirt.h | 27 +++
>> .../include/asm/paravirt_api_clock.h | 1 +
>> arch/loongarch/include/asm/smp.h | 31 ++--
>> arch/loongarch/include/uapi/asm/Kbuild | 2 -
>> arch/loongarch/kernel/Makefile | 1 +
>> arch/loongarch/kernel/irq.c | 24 +--
>> arch/loongarch/kernel/paravirt.c | 151 +++++++++++++++++
>> arch/loongarch/kernel/perf_event.c | 14 +-
>> arch/loongarch/kernel/smp.c | 62 ++++---
>> arch/loongarch/kernel/time.c | 12 +-
>> arch/loongarch/kvm/exit.c | 132 +++++++++++++--
>> arch/loongarch/kvm/vcpu.c | 94 ++++++++++-
>> arch/loongarch/kvm/vm.c | 11 ++
>> 22 files changed, 690 insertions(+), 102 deletions(-)
>> create mode 100644 arch/loongarch/include/asm/kvm_para.h
>> create mode 100644 arch/loongarch/include/asm/paravirt.h
>> create mode 100644 arch/loongarch/include/asm/paravirt_api_clock.h
>> delete mode 100644 arch/loongarch/include/uapi/asm/Kbuild
>> create mode 100644 arch/loongarch/kernel/paravirt.c
>>
>>
>> base-commit: 5eb4573ea63d0c83bf58fb7c243fc2c2b6966c02
>> --
>> 2.39.3
>>
>>
Powered by blists - more mailing lists