linux-kernel - Re: [PATCH] watchdog: nohz: don't run watchdog on nohz

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150402141527.GD175361@redhat.com>
Date:	Thu, 2 Apr 2015 10:15:27 -0400
From:	Don Zickus <dzickus@...hat.com>
To:	Chris Metcalf <cmetcalf@...hip.com>
Cc:	Ingo Molnar <mingo@...nel.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Andrew Jones <drjones@...hat.com>,
	chai wen <chaiw.fnst@...fujitsu.com>,
	Ulrich Obergfell <uobergfe@...hat.com>,
	Fabian Frederick <fabf@...net.be>,
	Aaron Tomlin <atomlin@...hat.com>,
	Ben Zhang <benzh@...omium.org>,
	Christoph Lameter <cl@...ux.com>,
	Frederic Weisbecker <fweisbec@...il.com>,
	Gilad Ben-Yossef <gilad@...yossef.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	open list <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] watchdog: nohz: don't run watchdog on nohz_full cores

On Thu, Apr 02, 2015 at 09:49:45AM -0400, Chris Metcalf wrote:
> >Can I ask how the NO_HZ_FULL technology works from userspace?  Is there a
> >system command that has to be sent?  How does the kernel know to turn off
> >ticks and trust userspace to do the right thing?
> 
> The NO_HZ_FULL option, when configured into the kernel, lets
> you boot with "nohz_full=1-15" (or whatever cpumask you like),
> typically in conjunction with "isolcpus=1-15".  At this point no tasks
> will run on those cores until explicitly placed there by affinity, and
> once there and running in userspace, the kernel will automatically
> get out of their way and not interrupt at all.  This lets those tasks
> run with 100.000% of the cpu, which is a requirement for many
> user-space device drivers running high throughput devices.
> (This is typically the use case for the tile architecture customers.)
> 
> So, other than a boot flag, there are no system commands or
> other APIs to deal with.

Ah, I am starting to understand your approach in the original patch better.

> 
> Part of the requirement, though, is that there can be only one task
> bound and runnable on that cpu, otherwise the kernel has to be
> involved to do the context-switching off of the scheduler tick.
> This is why having the standard watchdog kernel thread doesn't
> work in this context.

So, there is no preemption happening, which means the softlockup is rather
pointless.  Can interrupts be disabled or handled on that cpu?  I am trying
to see if the hardlockup detector becomes rather silly on those cpus too.

> 
> I continue to suspect that the right model here is to disable the
> watchdog specifically on the cores that the user has tagged with
> the nohz_full boot argument.  I agree that there might be a case
> to be made for leaving the watchdog conditionally (as suggested
> by Ingo) but it should be possible to have the watchdogs on
> the nohz_full cores be turned off completely if desired.

I think I might be slowly coming around to your thoughts.  I might request a
different patch though based on the answers above.  Maybe even create a
subset of the online cpus for the watchdog to work off of.  The watchdog
would copy the online cpu mask, mask off the nohz cpus and just function
that way.  It would print loud messages for each nohz cpu it was masking
off.

Then perhaps as a debug aid, expose a /proc/sys/kernel/watchdog_cpumask for
folks to modify in case they want to enable the watchdog on the nohz cpus.

Just some thoughts.

Cheers,
Don
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/