[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20201211180809.GA397355@yury-ThinkPad>
Date: Fri, 11 Dec 2020 10:08:09 -0800
From: Yury Norov <yury.norov@...il.com>
To: Thomas Gleixner <tglx@...utronix.de>
Cc: Pavel Machek <pavel@....cz>, Alex Belits <abelits@...vell.com>,
"nitesh@...hat.com" <nitesh@...hat.com>,
"frederic@...nel.org" <frederic@...nel.org>,
Prasun Kapoor <pkapoor@...vell.com>,
"linux-api@...r.kernel.org" <linux-api@...r.kernel.org>,
"davem@...emloft.net" <davem@...emloft.net>,
"trix@...hat.com" <trix@...hat.com>,
"mingo@...nel.org" <mingo@...nel.org>,
"catalin.marinas@....com" <catalin.marinas@....com>,
"rostedt@...dmis.org" <rostedt@...dmis.org>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"peterx@...hat.com" <peterx@...hat.com>,
"linux-arch@...r.kernel.org" <linux-arch@...r.kernel.org>,
"mtosatti@...hat.com" <mtosatti@...hat.com>,
"will@...nel.org" <will@...nel.org>,
"peterz@...radead.org" <peterz@...radead.org>,
"leon@...ebranch.com" <leon@...ebranch.com>,
"linux-arm-kernel@...ts.infradead.org"
<linux-arm-kernel@...ts.infradead.org>,
"pauld@...hat.com" <pauld@...hat.com>,
"netdev@...r.kernel.org" <netdev@...r.kernel.org>
Subject: Re: [PATCH v5 0/9] "Task_isolation" mode
On Sun, Dec 06, 2020 at 12:25:45AM +0100, Thomas Gleixner wrote:
> Pavel,
>
> On Sat, Dec 05 2020 at 21:40, Pavel Machek wrote:
> > So... what kind of guarantees does this aim to provide / what tasks it
> > is useful for?
> >
> > For real time response, we have other approaches.
>
> Depends on your requirements. Some problems are actually better solved
> with busy polling. See below.
>
> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...
>
> Applications which want to run as undisturbed as possible. There is
> quite a range of those:
>
> - Hardware in the loop simulation is today often done with that crude
> approach of "offlining" a CPU and then instead of playing dead
> jumping to a preloaded bare metal executable. That's a horrible hack
> and impossible to debug, but gives them the results they need to
> achieve. These applications are well optimized vs. cache and memory
> foot print, so they don't worry about these things too much and they
> surely don't run on SMI and BIOS value add inflicted machines.
>
> Don't even think about waiting for an interrupt to achieve what
> these folks are doing. So no, there are problems which a general
> purpose realtime OS cannot solve ever.
>
> - HPC computations on large data sets. While the memory foot print is
> large the access patterns are cache optimized.
>
> The problem there is that any unnecessary IPI, tick interrupt or
> whatever nuisance is disturbing the carefully optimized cache usage
> and alone getting rid of the timer interrupt gained them measurable
> performance. Even very low single digit percentage of runtime saving
> is valuable for these folks because the compute time on such beasts
> is expensive.
>
> - Realtime guests in KVM. With posted interrupts and a fully populated
> host side page table there is no point in running host side
> interrupts or IPIs for random accounting or whatever purposes as
> they affect the latency in the guest. With all the side effects
> mitigated and a properly set up guest and host it is possible to get
> to a zero exit situation after the bootup phase which means pretty
> much matching bare metal behaviour.
>
> Yes, you can do that with e.g. Jailhouse as well, but you lose lots
> of the fancy things KVM provides. And people care about these not
> just because they are fancy. They care because their application
> scenario needs them.
>
> There are more reasons why people want to be able to get as much
> isolation from the OS as possible but at the same time have a sane
> execution environment, debugging, performance monitoring and the OS
> provided protection mechanisms instead of horrible hacks.
>
> Isolation makes sense for a range of applications and there is no reason
> why Linux should not support them.
One good client for the task isolation is Open Data Plane. There are
even some code stubs supposed to enable isolation where needed.
> > If you want to guarantee performnace of the "isolated" task... I don't
> > see how that works. Other tasks on the system still compete for DRAM
> > bandwidth, caches, etc...
My experiments say that typical delay caused by dry IPI or syscall is
2000-20000 'ticks'. Typical delay caused by cache miss is 3-30 ticks.
To guarantee cache / memory bandwidth, one can use resctrl. Linux has
implementation of it for x86 only, but arm64 has support for for
resctrl on CPU side.
Thanks,
Yury
> Thanks,
>
> tglx
Powered by blists - more mailing lists