Date:	Fri, 15 Apr 2016 19:08:15 +0200
From:	"Luis R. Rodriguez" <mcgrof@...nel.org>
To:	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
Cc:	"Luis R. Rodriguez" <mcgrof@...nel.org>,
	Juergen Gross <jgross@...e.com>,
	Matt Fleming <matt@...eblueprint.co.uk>,
	Michael Chang <MChang@...e.com>, linux-kernel@...r.kernel.org,
	Jim Fehlig <jfehlig@...e.com>, Jan Beulich <JBeulich@...e.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	Daniel Kiper <daniel.kiper@...cle.com>, x86@...nel.org,
	Vojtěch Pavlík <vojtech@...e.cz>,
	Gary Lin <GLin@...e.com>, xen-devel@...ts.xenproject.org,
	Jeffrey Cheung <JCheung@...e.com>,
	Stefano Stabellini <stefano.stabellini@...citrix.com>,
	joeyli <jlee@...e.com>, Borislav Petkov <bp@...en8.de>,
	Joerg Roedel <joro@...tes.org>,
	Boris Ostrovsky <boris.ostrovsky@...cle.com>,
	Charles Arndol <carnold@...e.com>,
	Andrew Cooper <andrew.cooper3@...rix.com>,
	Julien Grall <julien.grall@....com>,
	Andy Lutomirski <luto@...capital.net>,
	David Vrabel <david.vrabel@...rix.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Roger Pau Monné <roger.pau@...rix.com>,
	Josh Triplett <josh@...htriplett.org>,
	Kees Cook <keescook@...omium.org>,
	Vitaly Kuznetsov <vkuznets@...hat.com>,
	Ard Biesheuvel <ard.biesheuvel@...aro.org>
Subject: Re: [Xen-devel] HVMLite / PVHv2 - using x86 EFI boot entry

On Thu, Apr 14, 2016 at 10:02:47PM -0400, Konrad Rzeszutek Wilk wrote:
> On Thu, Apr 14, 2016 at 10:56:19PM +0200, Luis R. Rodriguez wrote:
> > Are you telling me that HVMLite has no dead code issues ?
> 
> You said earlier that baremetal has dead code issue. Then by extensions
> _any_ execution path has dead code issues.

;)

> > Are you *sure* we have no dead code concerns with HVMLite ?
> > If there are dead code concerns are you sure there might not
> > be differences between KVM and HVMLite ? Should cpuid be used to
> > address differences ? Will that enable to distinguish between
> > hybrid versions of HVMLite ? Are we sure ?
> 
> HVMLite CPU semantics will be the same as what a baremetal CPU
> semantics are.
> 
> Platform wise it will be different - as in, instead of say
> having a speaker (to emulated it) or RTC clock (again, another
> thing to emulate), or say IDE controller (again, another
> thing to emulate), or Realtek network card (again, another
> thing to emulate) - it has none of those.
> 
> [Keep in mind 'another thing to emulate', means 'another
> @$@() thing in QEMU that could be a security bug']
> 
> So it differs from an consumer x86 platform in that it has
> none of the 'legacy' stuff. And it requires PV drivers to
> function. And since it requires PV drivers to function
> only OSes that have those can use this mode.

OK that was a long winded way to suggest dead code differences may really lie
in the implications of using some PV drivers.  That seems sensible to me and is
a good starting point to consider in the future. This is helpful thanks.

For KVM, it shall be pretty similar, however instead of PV drivers we'd be
dealing with emulated hardware and drivers using / being able to detect
emulation.

For both, the size of the dead code that is possible would grow depending on
the dependencies / libraries that these things use in comparison to bare metal.

Likewise other than this both will rely on hardware virtualization extensions
and this may implicate some dead code impact, it will depend on what run time
differences this causes on code and the dependencies on those components.

> > > > We cannot use cpuid early in asm code, I'm looking for something we
> > > 
> > > ?! Why!?
> > 
> > What existing code uses it? If there is nothing you are still certain
> > it should work ? Would that work for old PV guest as well BTW ?
> 
> Yeah. For HVM/HVMLite it traps to the hypervisor.
> 
> For old PV guests it is unwise to use it as it goes straight to
> the hardware (as PV guests run in ring3 - they are considered
> 'userspace' and the Intel nor AMD do not trap on 'cpuid' in ring3
> -unless  you run in an VMX container).

OK one heuristic we may need then for both KVM and HVMLite could be 'Are we
using hardware virtualization extensions', as that seems to implicate we can
then trust cpuid to zero-in on any further low level virtualization semantics
we may need and from what you are saying this should even work on asm code as
early as the sun rises.

> > > > Right, but I'm saying that its rather silly to be adding entry points if
> > > > all we want the code to do is copy the boot params for us. The design
> > > > doc requires a new entry, and likewise you'd need yet-another-entry if
> > > > HVMLite is thrown out the window and come back 5 years later after new
> > > > hardware solutions are in place and need to redesign HVMLite. Kind of
> > > 
> > > Why would you need to redesign HVMLite based on hardware solutions?
> > 
> > That's what happened to Xen PV, right ? Are we sure 5 years from now we won't
> > have any new hardware virtualization features that will just obsolete HVMLite?
> 
> There were no hardware virtualization when Xen PV came about.
> 
> If there is hardware virtualization that obsoletes HVMLite that means
> it would also obsolete KVM and HVM mode

Indeed.

> > > The entrace point and the CPU state are pretty well known - it is akin
> > > to what GRUB2 bootloader path is (protected mode).
> > > > where we are with PVH today. Likewise if other paravirtualization
> > > > developers want to support Linux and want to copy your strategy they'd
> > > > add yet-another-entry-point as well.
> > > > 
> > > > This is dumb.
> > > 
> > > You saying the EFI entry point is dumb? That instead the EFI
> > > firmware should understand Linux bootparams and booted that?
> > 
> > EFI is a standard. Xen is not. And since we are not talking about legacy
> 
> And is a standard something that has to come out of a committee?
> 
> If so, then Linux bootparams is not a standard. Nor is LILO bootup
> path.

My point was that it is very likely that OSes will implement booting
EFI, so if they were going to do that, and if this was going to be
streamlined on different architectures EFI as an entry point makes
sense to consider for future code as an entry point.

> > hardware in the future, EFI seems like a sensible option to consider for an
> > entry point. Specially given that it may mean that we can ultimately also help
> > unify more entry points on Linux in general. I'd prefer to consider using
> 
> <chokes>
> I can just see that. On non-EFI hardware GRUB2/SYSLINUX would use the EFI entry
> point and create an fake firmware.

That would be stupid, however it would also be stupid to not use EFI
if available if there were benefits from using it... so in context
the question here is if hardware supports EFI, would it be a sensible
option for HVMLite to use ?

> > EFI configuration tables instead of extending the x86 boot protocol.
> 
> What is that? Are you talking about EFI runtime services? Take a look
> at the EFI spec and see what you have to implement to emulate this.

Here's an example:

https://lkml.kernel.org/r/CAKv+Gu-=e4YiaR15+MWkazxH6s3ELnYj1BAV_PeuzhkLQpoNqA@mail.gmail.com

Its used to get scree_info on ARM.

> > 
> > > > > > If we had a single 'type' and 'custom data' passed to the kernel that
> > > > > > should suffice for the default Linux entry point to just pivot off
> > > > > > of that and do what it needs without more entry points. Once.
> > > > > 
> > > > > And what about ramdisk? What about multiple ramdisks?
> > > > > What about command line? All of that is what bootparams
> > > > > tries to unify on Linux. But 'bootparams' is unique to Linux,
> > > > > it does not exist on FreeBSD. Hence some stub code to transplant
> > > > > OS-agnostic simple data to OS-specific is neccessary.
> > > > 
> > > > If we had a Xen ABI option where *all* that I'm asking is you pass
> > > > first:
> > > > 
> > > >   a) hypervisor type
> > > 
> > > Why can't you use cpuid.
> > 
> > I'll evaluate that.
> > 
> > > >   b) custom data pointer
> > > 
> > > What is this custom data pointer you speak of?
> > 
> > For Xen this is the xen_start_info, the structure that Xen stuffs in
> > a copy of its version of what we need to fill the boot_params.
> 
> Ok, but that is what we do in some way provide.
> 
> I am lost here. You seem to saying you want something that is
> already there?

Its not done in a way that avoids yet-another-entry point. I am looking
to see if we can instead let the OS agnostic data still be passed but
let the kernel stick to its existing entry points without adding
another entry point just for HVMLite.

> > > > We'd be able to avoid adding *any* entry point and just address
> > > > the requirements as I noted with pre / post stubs for the type.
> > > 
> > > But you need some entry point to call into Linux. Are you
> > > suggesting to use the existing ones? No, the existing one
> > > wouldn't understand this.
> > 
> > If we used the boot_parms, yes it would be possible.
> 
> ...OS agnostic... they are not.

Right... 

> > > > This would require an x86 boot protocol bump, but all the issues
> > > > creeping up randomly I think that's worth putting on the table now.
> > > 
> > > Aaaah, so you are saying expand the bootparams. In other words
> > > make Xen ABI call into Linux using the bootparams structure, similar
> > > to how GRUB2 does it.
> > > 
> > > How is that OS agnostic?
> > 
> > That's an issue, I understand. EFI is OS agnostic though.
> > 
> > > > And maybe we don't want it to be hypervisor specific, perhaps there are other
> > > > *needs* for custom pre-post startup_32()/startup_64() stubs.
> > > 
> > > Multiboot?
> > 
> > Can you elaborate?
> 
> Google Multiboot specification.

Neat thanks. Hrm...

> > > > To avoid extending boot_params further I figured perhaps we can look
> > > > at EFI as another option instead. If we are going to drop all legacy
> > > 
> > > But EFI support is _huge_.
> > 
> > I get the sense now. Perhaps we should explore to what extent now really
> > at the Hackathon.
> 
> Print out the EFI spec and carry it on the plane. The plane will tilt
> to one side when trying to take off.

Maybe I'll take Xen code as well to balance it all out.

  Luis

Powered by blists - more mailing lists