linux-kernel - Re: [PATCH v6 09/10] hisi_acc_vfio_pci: Add support for VFIO live migration

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20220301203938.GY219866@nvidia.com>
Date:   Tue, 1 Mar 2022 16:39:38 -0400
From:   Jason Gunthorpe <jgg@...dia.com>
To:     Alex Williamson <alex.williamson@...hat.com>
Cc:     Shameerali Kolothum Thodi <shameerali.kolothum.thodi@...wei.com>,
        "kvm@...r.kernel.org" <kvm@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-crypto@...r.kernel.org" <linux-crypto@...r.kernel.org>,
        "cohuck@...hat.com" <cohuck@...hat.com>,
        "mgurtovoy@...dia.com" <mgurtovoy@...dia.com>,
        "yishaih@...dia.com" <yishaih@...dia.com>,
        Linuxarm <linuxarm@...wei.com>,
        liulongfang <liulongfang@...wei.com>,
        "Zengtao (B)" <prime.zeng@...ilicon.com>,
        Jonathan Cameron <jonathan.cameron@...wei.com>,
        "Wangzhou (B)" <wangzhou1@...ilicon.com>
Subject: Re: [PATCH v6 09/10] hisi_acc_vfio_pci: Add support for VFIO live
 migration

On Tue, Mar 01, 2022 at 12:30:47PM -0700, Alex Williamson wrote:
> Wouldn't it make more sense if initial-bytes started at QM_MATCH_SIZE
> and dirty-bytes was always sizeof(vf_data) - QM_MATCH_SIZE?  ie. QEMU
> would know that it has sizeof(vf_data) - QM_MATCH_SIZE remaining even
> while it's getting ENOMSG after reading QM_MATCH_SIZE bytes of data.

The purpose of this ioctl is to help userspace guess when moving on to
STOP_COPY is a good idea ie when the device has done almost all the
work it is going to be able to do in PRE_COPY. ENOMSG is a similar
indicator.

I expect all devices to have some additional STOP_COPY trailer_data in
addition to their PRE_COPY initial_data and dirty_data

There is a choice to make if we report the trailer_data during
PRE_COPY or not. As this is all estimates, it doesn't matter unless
the trailer_data is very big.

Having all devices trend toward a 0 dirty_bytes to say they are are
done all the pre-copy they can do makes sense from an API
perspective. If one device trends toward 10MB due to a big
trailer_data and one trends toward 0 bytes, how will qemu consistently
decide when best to trigger STOP_COPY? It makes the API less useful.

So, I would not include trailer_data in the dirty_bytes.

Estimating when to move on to STOP_COPY and trying to enforce a SLA on
STOP_COPY are different tasks and will probably end up with different
interfaces.

I still think the right way to approach the SLA is to inform the
driver what the permitted time and data size target is for STOP_COPY
and the driver can proceed or not based on its own internal
calculation.

> useful yet and you don't want to add dead kernel code, then let's
> define that this ioctl is only available in the PRE_COPY* states and
> returns -errno in the STOP_COPY state.

I'm OK with that, in acc it is done by checking migf->total_bytes >
QM_MATCH_SIZE during the read fop

> devices in STOP_COPY and let's also define if there's actually anything
> userspace can infer about remaining STOP_COPY data size while in
> PRE_COPY* via this ioctl.  For example, is dirty-bytes zero or the
> remaining data structure size?

If we keep it then I would say it doesn't matter, userspace has to sum
the two values to get the total remaining length estimate, it is just
a bit quirky.

Jason