linux-kernel - Re: [RFC PATCH v1 00/10] Add AMD SEV guest live migration support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <eee5e112-89bd-331b-6875-34c0712de4cb@amd.com>
Date:   Wed, 24 Apr 2019 21:32:39 +0000
From:   "Singh, Brijesh" <brijesh.singh@....com>
To:     Steve Rutherford <srutherford@...gle.com>
CC:     "Singh, Brijesh" <brijesh.singh@....com>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "qemu-devel@...gnu.org" <qemu-devel@...gnu.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Radim Krčmář <rkrcmar@...hat.com>,
        Joerg Roedel <joro@...tes.org>, Borislav Petkov <bp@...e.de>,
        "Lendacky, Thomas" <Thomas.Lendacky@....com>,
        "x86@...nel.org" <x86@...nel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH v1 00/10] Add AMD SEV guest live migration support



On 4/24/19 2:15 PM, Steve Rutherford wrote:
> On Wed, Apr 24, 2019 at 9:10 AM Singh, Brijesh <brijesh.singh@....com> wrote:
>>
>> The series add support for AMD SEV guest live migration commands. To protect the
>> confidentiality of an SEV protected guest memory while in transit we need to
>> use the SEV commands defined in SEV API spec [1].
>>
>> SEV guest VMs have the concept of private and shared memory. Private memory
>> is encrypted with the guest-specific key, while shared memory may be encrypted
>> with hypervisor key. The commands provided by the SEV FW are meant to be used
>> for the private memory only. The patch series introduces a new hypercall.
>> The guest OS can use this hypercall to notify the page encryption status.
>> If the page is encrypted with guest specific-key then we use SEV command during
>> the migration. If page is not encrypted then fallback to default.
>>
>> The patch adds a new ioctl KVM_GET_PAGE_ENC_BITMAP. The ioctl can be used
>> by the qemu to get the page encrypted bitmap. Qemu can consult this bitmap
>> during the migration to know whether the page is encrypted.
>>
>> [1] https://developer.amd.com/wp-content/resources/55766.PDF
>>
>> The series is tested with the Qemu, I am in process of cleaning
>> up the Qemu code and will submit soon.
>>
>> While implementing the migration I stumbled on the follow question:
>>
>> - Since there is a guest OS changes required to support the migration,
>>    so how do we know whether guest OS is updated? Should we extend KVM
>>    capabilities/feature bits to check this?
>>
>> TODO:
>>   - add an ioctl to build encryption bitmap. The encryption bitmap is built during
>>     the guest bootup/execution. We should provide an ioctl so that destination
>>     can build the bitmap as it receives the pages.
>>   - reset the bitmap on guest reboot.
>>
>> The complete tree with patch is available at:
>> https://github.com/codomania/kvm/tree/sev-migration-rfc-v1
>>
>> Cc: Thomas Gleixner <tglx@...utronix.de>
>> Cc: Ingo Molnar <mingo@...hat.com>
>> Cc: "H. Peter Anvin" <hpa@...or.com>
>> Cc: Paolo Bonzini <pbonzini@...hat.com>
>> Cc: "Radim Krčmář" <rkrcmar@...hat.com>
>> Cc: Joerg Roedel <joro@...tes.org>
>> Cc: Borislav Petkov <bp@...e.de>
>> Cc: Tom Lendacky <thomas.lendacky@....com>
>> Cc: x86@...nel.org
>> Cc: kvm@...r.kernel.org
>> Cc: linux-kernel@...r.kernel.org
>>
>> Brijesh Singh (10):
>>    KVM: SVM: Add KVM_SEV SEND_START command
>>    KVM: SVM: Add KVM_SEND_UPDATE_DATA command
>>    KVM: SVM: Add KVM_SEV_SEND_FINISH command
>>    KVM: SVM: Add support for KVM_SEV_RECEIVE_START command
>>    KVM: SVM: Add KVM_SEV_RECEIVE_UPDATE_DATA command
>>    KVM: SVM: Add KVM_SEV_RECEIVE_FINISH command
>>    KVM: x86: Add AMD SEV specific Hypercall3
>>    KVM: X86: Introduce KVM_HC_PAGE_ENC_STATUS hypercall
>>    KVM: x86: Introduce KVM_GET_PAGE_ENC_BITMAP ioctl
>>    mm: x86: Invoke hypercall when page encryption status is changed
>>
>>   .../virtual/kvm/amd-memory-encryption.rst     | 116 ++++
>>   Documentation/virtual/kvm/hypercalls.txt      |  14 +
>>   arch/x86/include/asm/kvm_host.h               |   3 +
>>   arch/x86/include/asm/kvm_para.h               |  12 +
>>   arch/x86/include/asm/mem_encrypt.h            |   3 +
>>   arch/x86/kvm/svm.c                            | 560 +++++++++++++++++-
>>   arch/x86/kvm/vmx/vmx.c                        |   1 +
>>   arch/x86/kvm/x86.c                            |  17 +
>>   arch/x86/mm/mem_encrypt.c                     |  45 +-
>>   arch/x86/mm/pageattr.c                        |  15 +
>>   include/uapi/linux/kvm.h                      |  51 ++
>>   include/uapi/linux/kvm_para.h                 |   1 +
>>   12 files changed, 834 insertions(+), 4 deletions(-)
>>
>> --
>> 2.17.1
>>
> 
> What's the back-of-the-envelope marginal cost of transferring a 16kB
> region from one host to another? I'm interested in what the end to end
> migration perf changes look like for this. If you have measured
> migration perf, I'm interested in that also.
> 

I have not done a complete performance analysis yet! From the qemu
QMP prompt (query-migration) I am getting ~8mbps throughput from
one host to another (this is with 4kb regions). I have been told
that increasing the transfer size from 4kb -> 16kb may not give a
huge performance gain because at FW level they still operating
on 4kb blocks. There is possibility that future FW updates may
give a bit better performance on 16kb size.

-Brijesh