Message-ID: <20160414010131.GA21510@localhost.localdomain>
Date:	Wed, 13 Apr 2016 21:01:32 -0400
From:	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>
To:	"Luis R. Rodriguez" <mcgrof@...nel.org>
Cc:	Juergen Gross <jgross@...e.com>,
	Matt Fleming <matt@...eblueprint.co.uk>,
	Michael Chang <MChang@...e.com>, linux-kernel@...r.kernel.org,
	Jim Fehlig <jfehlig@...e.com>, Jan Beulich <JBeulich@...e.com>,
	"H. Peter Anvin" <hpa@...or.com>,
	Daniel Kiper <daniel.kiper@...cle.com>, x86@...nel.org,
	Vojtěch Pavlík <vojtech@...e.cz>,
	Gary Lin <GLin@...e.com>, xen-devel@...ts.xenproject.org,
	Jeffrey Cheung <JCheung@...e.com>,
	Stefano Stabellini <stefano.stabellini@...citrix.com>,
	joeyli <jlee@...e.com>, Borislav Petkov <bp@...en8.de>,
	Boris Ostrovsky <boris.ostrovsky@...cle.com>,
	Charles Arndol <carnold@...e.com>,
	Andrew Cooper <andrew.cooper3@...rix.com>,
	Julien Grall <julien.grall@....com>,
	Andy Lutomirski <luto@...capital.net>,
	David Vrabel <david.vrabel@...rix.com>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Roger Pau Monné <roger.pau@...rix.com>
Subject: Re: [Xen-devel] HVMLite / PVHv2 - using x86 EFI boot entry

On Thu, Apr 14, 2016 at 12:23:17AM +0200, Luis R. Rodriguez wrote:
> On Wed, Apr 13, 2016 at 05:08:01PM -0400, Konrad Rzeszutek Wilk wrote:
> > On Wed, Apr 13, 2016 at 10:40:55PM +0200, Luis R. Rodriguez wrote:
> > > On Wed, Apr 13, 2016 at 02:56:29PM -0400, Konrad Rzeszutek Wilk wrote:
> > > > On Wed, Apr 13, 2016 at 08:29:51PM +0200, Luis R. Rodriguez wrote:
> > > > > On Mon, Apr 11, 2016 at 07:12:08AM +0200, Juergen Gross wrote:
> > > > > 
> > > > > > What would be gained by using the same entry but having two different boot
> > > > > > paths after it?
> > > > > 
> > > > > Its a good question. In summary for me it would be the push for sharing more
> > > > > code and the push for semantics on early boot to address differences
> > > > > proactively, and ultimately it may enable us to help bring closer the old PV
> > > > > boot path closer.
> > > > 
> > > > But why? We want to kill PV (eventually).
> > > 
> > > Yeah yeah, but its still there, and we'll have to live with it for
> > > at least minimum 5 years I hear. Part of my interest is to see to it
> > > that this path gets less disruption and issues, and we also address
> > > dead code issues which pvops simply folded under the rug. The dead code
> > > concerns may exist still for hvmlite, so unless someone is willing
> > > to make a bold claim there is none, its something to consider.
> > 
> > What is this dead code you speak of? Is it MTRR? Is early path code
> > that PV misses (like KASL or other?)
> 
> Kasan is dead code to Xen. If you boot x86 Xen with Kasan enabled

For Xen PV guests,
> Xen explodes. Quick question, will Kasan not explode with HVMLite ?

.. but for HVMLite of Xen HVM guest Kasan will run.
> 
> MTRR used to be dead code concern but since we have vetted most of that code
> now we are pretty certain that code should never run now.
> 
> KASLR may be -- not sure as I  haven't vetted that, but from
> what I have loosely heard maybe.
> 
> VGA code will be dead code for HVMlite for sure as the design doc
> says it will not run VGA, the ACPI flag will be set but the check
> for that is not yet on Linux. That means the VGA Linux code will
> be there but we have no way to ensure it will not run nor that
> anything will muck with it.

<shrugs> The worst it will do is try to read non-existent registers.
The VGA code should be able to handle failures like that and
not initialize itself when the hardware is dead (or non-existent).
> 
> To be clear -- dead code concerns still exist even without
> virtualization solutions, its just that with virtualization
> this stuff comes up more and there has been no proactive
> measures to address this. The question of semantics here is
> to see to what extent we need earlier boot code annotations
> to ensure we address semantics proactively.

I think what you mean by dead code is another word for
hardware test coverage?
> 
> > The entrace point in Linux "proper" is startup_32 or startup_64 - the same
> > path that EFI uses.
> > 
> > If you were to draw this (very simplified):
> > 
> > a)- GRUB2 ---------------------\ (creates an bootparam structure)
> >                                 \
> >                                  +---- startup_32 or startup_64
> > b) EFI -> Linux EFI stub -------/
> >        (creates bootparm)      /
> > c) GRUB2-EFI  -> Linux EFI----/
> >                stub         /
> > d) HVMLite ----------------/
> >       (creates bootparm)
> 
> b) and d) might be able to share paths there...

No idea. You would have to look in the assembler code to
figure that out.

> d) still has its own entry, it does more than create boot params.

d) purpose is to create boot params. It may do more as nobody likes
to muck in assembler and make bootparams from within assembler.

> 
> > (I am not sure about the c) - I would have to look in source to
> > be source). There is also LILO in this, but I am not even sure if
> > works anymore.
> > 
> > 
> > What you have is that every entry point creates the bootparams
> > and ends up calling startup_X. The startup_64 then hit the rest
> > of the kernel. The startp_X code is the one that would setup
> > the basic pagetables, segments, etc.
> 
> Sure.. a full diagram should include both sides and how when using
> a custom entry one runs the risk of skipping a lot of code setup.

But it does not skip a lot of code setup. It starts exactly
at the same code startup that _all_ bootstraping code start at.

> There is that and as others have pointed out how certain guests types
> are assumed to not have certain peripherals, and we have no idea
> to ensure certain old legacy code may not ever run or be accessed
> by drivers.

Ok, but that is not at code setup. That is later - when device
drivers are initialized. This no different than booting on
some hardware with missing functionality. ACPI, PCI and PnP
PnP are set there to help OSes discover this.
> 
> > > How we address semantics then is *very* important to me.
> > 
> > Which semantics? How the CPU is going to be at startup_X ? Or
> > how the CPU is going to be when EFI firmware invokes the EFI stub?
> > Or when GRUB2 loads Linux?
> 
> What hypervisor kicked me and what guest type I am.

cpuid software flags have that - and that semantics has been 
there for eons.
> 
> Let me elaborate more below.
> 
> > That (those bootloaders) is clearly defined. The URL I provided
> > mentions the HVMLite one. The Documentation/x86/boot.c mentions
> > what the semantics are to expected when providing an bootstrap
> > (which is what HVMLitel stub code in Linux would write against -
> > and what EFI stub code had been written against too).
> > > 
> > > > > I'll elaborate on this but first let's clarify why a new entry is used for
> > > > > HVMlite to start of with:
> > > > > 
> > > > >   1) Xen ABI has historically not wanted to set up the boot params for Linux
> > > > >      guests, instead it insists on letting the Linux kernel Xen boot stubs fill
> > > > >      that out for it. This sticking point means it has implicated a boot stub.
> > > > 
> > > > 
> > > > Which is b/c it has to be OS agnostic. It has nothing to do 'not wanting'.
> > > 
> > > It can still be OS agnostic and pass on type and custom data pointer.
> > 
> > Sure. It has that (it MUST otherwise how else would you pass data).
> > It is documented as well http://xenbits.xen.org/docs/unstable/hypercall/x86_64/include,public,xen.h.html#incontents_startofday
> > (see " Start of day structure passed to PVH guests in %ebx.")
> 
> The design doc begs for a custom OS entry point though.

That is what the ELF Note has.
> If we had a single 'type' and 'custom data' passed to the kernel that
> should suffice for the default Linux entry point to just pivot off
> of that and do what it needs without more entry points. Once.

And what about ramdisk? What about multiple ramdisks?
What about command line? All of that is what bootparams
tries to unify on Linux. But 'bootparams' is unique to Linux,
it does not exist on FreeBSD. Hence some stub code to transplant
OS-agnostic simple data to OS-specific is neccessary.
> 
>   Luis

Powered by blists - more mailing lists