[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <37D7C6CF3E00A74B8858931C1DB2F07753713AAD@SHSMSX103.ccr.corp.intel.com>
Date: Wed, 28 Jun 2017 14:32:12 +0000
From: "Liang, Kan" <kan.liang@...el.com>
To: Michal Hocko <mhocko@...nel.org>
CC: "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"dzickus@...hat.com" <dzickus@...hat.com>,
"mingo@...nel.org" <mingo@...nel.org>,
"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
"babu.moger@...cle.com" <babu.moger@...cle.com>,
"atomlin@...hat.com" <atomlin@...hat.com>,
"prarit@...hat.com" <prarit@...hat.com>,
"torvalds@...ux-foundation.org" <torvalds@...ux-foundation.org>,
"peterz@...radead.org" <peterz@...radead.org>,
"tglx@...utronix.de" <tglx@...utronix.de>,
"eranian@...gle.com" <eranian@...gle.com>,
"acme@...hat.com" <acme@...hat.com>,
"ak@...ux.intel.com" <ak@...ux.intel.com>,
"stable@...r.kernel.org" <stable@...r.kernel.org>
Subject: RE: [PATCH] kernel/watchdog: fix spurious hard lockups
> > > > From: Kan Liang <Kan.liang@...el.com>
> > > >
> > > > Some users reported spurious NMI watchdog timeouts.
> > > >
> > > > We now have more and more systems where the Turbo range is wide
> > > enough
> > > > that the NMI watchdog expires faster than the soft watchdog timer
> > > > that updates the interrupt tick the NMI watchdog relies on.
> > >
> > > AFAIR the watchdog doesn't rely on deferred timers so this would
> > > suggest that a standard hrtimer can expire much later than programmed,
> right?
> >
> > The softlockup watchdog relies on hrtimers.
> > The hardlockup watchdog (NMI watchdog) relies on perf subsystem and
> > using unhalted CPU cycles.
> > When the softlockup watchdog expires, it updates the hrtimer_interrupts.
> > When the NMI watchdog expires, it will check the hrtimer_interrupts,
> > and determine if it's a hardlockup.
> > The design was to make the softlockup watchdog runs with 2.5 times the
> > rate of NMI watchdog. So it guarantees that the hrtimer_interrupts is
> > updated before the NMI watchdog expires.
> > That works well if Turbo-Mode is disabled.
> > However, when Turbo-Mode is enabled, unhalted CPU cycles might run
> > much faster than expected, even faster than softlockup watchdog.
> > So the softlockup watchdog will not get a chance to update the
> > hrtimer_interrupts, which will trigger false positives.
>
> So it is not the hrtimer which doesn't fire but rather the NMI events fire too
> quickly, right?
Yes, the NMI fires too quickly.
Thanks,
Kan
Powered by blists - more mailing lists