linux-kernel - Re: [PATCH v2] watchdog/hardlockup: Avoid large stack frames in watchdog_hardlockup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ZMtbvscFCtu9WRCe@dhcp22.suse.cz>
Date:   Thu, 3 Aug 2023 09:48:14 +0200
From:   Michal Hocko <mhocko@...e.com>
To:     Douglas Anderson <dianders@...omium.org>
Cc:     Andrew Morton <akpm@...ux-foundation.org>,
        Petr Mladek <pmladek@...e.com>,
        kernel test robot <lkp@...el.com>,
        Lecopzer Chen <lecopzer.chen@...iatek.com>,
        Pingfan Liu <kernelfans@...il.com>,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2] watchdog/hardlockup: Avoid large stack frames in
 watchdog_hardlockup_check()

On Wed 02-08-23 12:26:00, Douglas Anderson wrote:
> After commit 77c12fc95980 ("watchdog/hardlockup: add a "cpu" param to
> watchdog_hardlockup_check()") we started storing a `struct cpumask` on
> the stack in watchdog_hardlockup_check(). On systems with
> CONFIG_NR_CPUS set to 8192 this takes up 1K on the stack. That
> triggers warnings with `CONFIG_FRAME_WARN` set to 1024.
> 
> Instead of putting this `struct cpumask` on the stack, we'll allocate
> it on the heap whenever userspace tells us that they want to backtrace
> all CPUs upon a hardlockup.
> 
> NOTE: the reason that this mask is even needed is to make sure that we
> can print the hung CPU first, which makes the logs much easier to
> understand.
> 
> Fixes: 77c12fc95980 ("watchdog/hardlockup: add a "cpu" param to watchdog_hardlockup_check()")
> Reported-by: kernel test robot <lkp@...el.com>
> Closes: https://lore.kernel.org/r/202307310955.pLZDhpnl-lkp@intel.com
> Signed-off-by: Douglas Anderson <dianders@...omium.org>
> ---
> 
> Changes in v2:
> - Allocate space when userspace requests all cpus be backtraced.
> 
>  kernel/watchdog.c | 44 ++++++++++++++++++++++++++++++++++----------
>  1 file changed, 34 insertions(+), 10 deletions(-)
> 
> diff --git a/kernel/watchdog.c b/kernel/watchdog.c
> index be38276a365f..25d5627a6580 100644
> --- a/kernel/watchdog.c
> +++ b/kernel/watchdog.c
> @@ -93,6 +93,8 @@ static DEFINE_PER_CPU(bool, watchdog_hardlockup_warned);
>  static DEFINE_PER_CPU(bool, watchdog_hardlockup_touched);
>  static unsigned long watchdog_hardlockup_all_cpu_dumped;
>  
> +static struct cpumask *hardlockup_backtrace_mask;
> +
>  notrace void arch_touch_nmi_watchdog(void)
>  {
>  	/*
> @@ -106,6 +108,29 @@ notrace void arch_touch_nmi_watchdog(void)
>  }
>  EXPORT_SYMBOL(arch_touch_nmi_watchdog);
>  
> +static int hardlockup_all_cpu_backtrace_proc_handler(struct ctl_table *table, int write,
> +		  void *buffer, size_t *lenp, loff_t *ppos)
> +{
> +	int ret;
> +
> +	ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
> +
> +	/*
> +	 * Only allocate memory for the backtrace mask if userspace actually
> +	 * wants to trace all CPUs since this can take up 1K of space on a
> +	 * system with CONFIG_NR_CPUS=8192.
> +	 */
> +	if (sysctl_hardlockup_all_cpu_backtrace && !hardlockup_backtrace_mask) {
> +		hardlockup_backtrace_mask =
> +			   kzalloc(sizeof(*hardlockup_backtrace_mask), GFP_KERNEL);
> +	} else if (!sysctl_hardlockup_all_cpu_backtrace && hardlockup_backtrace_mask) {
> +		kfree(hardlockup_backtrace_mask);
> +		hardlockup_backtrace_mask = NULL;
> +	}

While unlikely, this can race with the consumer and cause either
use-after-free or NULL ptr deref.
-- 
Michal Hocko
SUSE Labs