Message-ID: <YshvnodeqmJV6uIJ@casper.infradead.org>
Date:   Fri, 8 Jul 2022 18:55:42 +0100
From:   Matthew Wilcox <willy@...radead.org>
To:     Nadav Amit <namit@...are.com>
Cc:     Ajay Kaher <akaher@...are.com>, Bjorn Helgaas <helgaas@...nel.org>,
        "bhelgaas@...gle.com" <bhelgaas@...gle.com>,
        "tglx@...utronix.de" <tglx@...utronix.de>,
        "mingo@...hat.com" <mingo@...hat.com>,
        "bp@...en8.de" <bp@...en8.de>,
        "dave.hansen@...ux.intel.com" <dave.hansen@...ux.intel.com>,
        "x86@...nel.org" <x86@...nel.org>, "hpa@...or.com" <hpa@...or.com>,
        "linux-pci@...r.kernel.org" <linux-pci@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "rostedt@...dmis.org" <rostedt@...dmis.org>,
        Srivatsa Bhat <srivatsab@...are.com>,
        "srivatsa@...il.mit.edu" <srivatsa@...il.mit.edu>,
        Alexey Makhalov <amakhalov@...are.com>,
        Anish Swaminathan <anishs@...are.com>,
        Vasavi Sirnapalli <vsirnapalli@...are.com>,
        "er.ajay.kaher@...il.com" <er.ajay.kaher@...il.com>
Subject: Re: [PATCH] MMIO should have more priority then IO

On Fri, Jul 08, 2022 at 04:45:00PM +0000, Nadav Amit wrote:
> On Jul 8, 2022, at 5:56 AM, Matthew Wilcox <willy@...radead.org> wrote:
> 
> > And looking at the results above, it's not so much the PIO vs MMIO
> > that makes a difference, it's the virtualisation. A mmio access goes
> > from 269ns to 85us. Rather than messing around with preferring MMIO
> > over PIO for config space, having an "enlightenment" to do config
> > space accesses would be a more profitable path.
> 
> I am unfamiliar with the motivation for this patch, but I just wanted to
> briefly regard the advice about enlightments.
> 
> “enlightenment”, AFAIK, is Microsoft’s term for "para-virtualization", so
> let’s regard the generic term. I think that you consider the bare-metal
> results as the possible results from a paravirtual machine, which is mostly
> wrong. Para-virtualization usually still requires a VM-exit and for the most
> part the hypervisor/host runs similar code for MMIO/hypercall (conceptually;
> the code of paravirtual and fully-virtual devices is often different, but
> IIUC, this is not what Ajay measured).
> 
> Para-virtualization could have *perhaps* helped to reduce the number of
> PIO/MMIO and improve performance this way. If, for instance, all the
> PIO/MMIO are done during initialization, a paravirtual interface can be use
> to batch them together, and that would help. But it is more complicated to
> get a performance benefit from paravirtualization if the PIO/MMIO accesses
> are “spread”, for instance, done after each interrupt.

What kind of lousy programming interface requires you to do a config
space access after every interrupt?  This is looney-tunes.

You've used a lot of words to not answer the question that was so
important that I asked it twice.  What's the use case, what's the
workload that would benefit from this patch?

> Para-virtauilzation and full-virtualization both have pros and cons.
> Para-virtualization is many times more efficient, but requires the VM to
> have dedicated device drivers for the matter. Try to run a less-common OS
> than Linux and it would not work since the OS would not have drivers for the
> paras-virtual devices. And even if you add support today for a para-virtual
> devices, there are many deployed OSes that do not have such support, and you
> would not be able to run them in a VM.
> 
> Regardless to virtualization, Ajay’s results show PIO is slower on
> bare-metal, and according to his numbers by 165ns, which is significant.
> Emulating PIO in hypervisors on x86 is inherently more complex than MMIO, so
> the results he got would most likely happen on all hypervisors.
> 
> tl;dr: Let’s keep this discussion focused and put paravirtualization aside.
> It is not a solution for all the problems in the world.

Powered by blists - more mailing lists