linux-kernel - Re: [PATCH] sched: Warn on long periods of pending need

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210317082550.GA3881262@gmail.com>
Date:   Wed, 17 Mar 2021 09:25:50 +0100
From:   Ingo Molnar <mingo@...nel.org>
To:     Josh Don <joshdon@...gle.com>
Cc:     Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Juri Lelli <juri.lelli@...hat.com>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Steven Rostedt <rostedt@...dmis.org>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Luis Chamberlain <mcgrof@...nel.org>,
        Kees Cook <keescook@...omium.org>,
        Iurii Zaikin <yzaikin@...gle.com>,
        linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
        David Rientjes <rientjes@...gle.com>,
        Oleg Rombakh <olegrom@...gle.com>, Paul Turner <pjt@...gle.com>
Subject: Re: [PATCH] sched: Warn on long periods of pending need_resched


* Josh Don <joshdon@...gle.com> wrote:

> From: Paul Turner <pjt@...gle.com>
> 
> CPU scheduler marks need_resched flag to signal a schedule() on a
> particular CPU. But, schedule() may not happen immediately in cases
> where the current task is executing in the kernel mode (no
> preemption state) for extended periods of time.
> 
> This patch adds a warn_on if need_resched is pending for more than the
> time specified in sysctl resched_latency_warn_ms. Monitoring is done via
> the tick and the accuracy is hence limited to jiffy scale. This also
> means that we won't trigger the warning if the tick is disabled.

Looks useful.

> If resched_latency_warn_ms is set to the default value, only one warning
> will be produced per boot.

Looks like a value hack, should probably be a separate flag, 
defaulting to warn-once.

> This warning only exists under CONFIG_SCHED_DEBUG. If it goes off, it is
> likely that there is a missing cond_resched() somewhere.

CONFIG_SCHED_DEBUG is default-y, so most distros have it enabled.

> +/*
> + * Print a warning if need_resched is set for at least this long. At the
> + * default value, only a single warning will be printed per boot.
> + *
> + * Values less than 2 disable the feature.
> + *
> + * A kernel compiled with CONFIG_KASAN tends to run more slowly on average.
> + * Increase the need_resched timeout to reduce false alarms.
> + */
> +#ifdef CONFIG_KASAN
> +#define RESCHED_DEFAULT_WARN_LATENCY_MS 101
> +#define RESCHED_BOOT_QUIET_SEC 600
> +#else
> +#define RESCHED_DEFAULT_WARN_LATENCY_MS 51
> +#define RESCHED_BOOT_QUIET_SEC 300
>  #endif
> +int sysctl_resched_latency_warn_ms = RESCHED_DEFAULT_WARN_LATENCY_MS;
> +#endif /* CONFIG_SCHED_DEBUG */

I'd really just make this a single value - say 100 or 200 msecs.

> +static inline void resched_latency_warn(int cpu, u64 latency)
> +{
> +	static DEFINE_RATELIMIT_STATE(latency_check_ratelimit, 60 * 60 * HZ, 1);
> +
> +	WARN(__ratelimit(&latency_check_ratelimit),
> +	     "CPU %d: need_resched set for > %llu ns (%d ticks) "
> +	     "without schedule\n",
> +	     cpu, latency, cpu_rq(cpu)->ticks_without_resched);
> +}

Could you please put the 'sched:' prefix into scheduler warnings. 
Let's have a bit of a namespace structure in new warnings.

Thanks,

	Ingo