Message-ID: <aRncJo1mA5Zk77Hr@U-2FWC9VHC-2323.local>
Date: Sun, 16 Nov 2025 22:13:58 +0800
From: Feng Tang <feng.tang@...ux.alibaba.com>
To: Lance Yang <lance.yang@...ux.dev>,
	Andrew Morton <akpm@...ux-foundation.org>
Cc: Petr Mladek <pmladek@...e.com>, Steven Rostedt <rostedt@...dmis.org>,
	Lance Yang <ioworker0@...il.com>, linux-kernel@...r.kernel.org,
	Jonathan Corbet <corbet@....net>, paulmck@...nel.org,
	lirongqing@...du.com, leonylgao@...cent.com
Subject: Re: [PATCH v2 2/4] hung_task: Add hung_task_sys_info sysctl to dump
 sys info on task-hung

On Sun, Nov 16, 2025 at 09:22:43PM +0800, Lance Yang wrote:
> > > Looking at the history:
> > > 
> > > 1) Commit ("hung_task: ignore hung_task_warnings when hung_task_panic
> > > is enabled")[1] ensured that hung task information is always dumped
> > > when a panic is configured, even if the warning counter is exhausted.
> > > 
> > > 2) Later, commit ("hung_task: panic when there are more than N hung
> > > tasks at the same time")[2] refined the logic to trigger a panic based
> > > on the number of hung tasks found in a single scan.
> > > 
> > > To stay consistent with the established behavior, I think we should
> > > continue to dump the information for hung tasks as long as
> > > sysctl_hung_task_panic is enabled :)
> > > 
> > > [1] https://lore.kernel.org/all/20240613033159.3446265-1-leonylgao@gmail.com
> > > [2] https://lore.kernel.org/all/20251015063615.2632-1-lirongqing@baidu.com
> > > [...]
> > 
> > Aha, Petr asked similar question during his review. Thanks for the catch!
> > 
> > How about following fixup patch to restore that part of logic?
> > 
> > Thanks,
> > Feng
> > 
> > ---
> > diff --git a/kernel/hung_task.c b/kernel/hung_task.c
> > index 5b3a7785d3a2..d2254c91450b 100644
> > --- a/kernel/hung_task.c
> > +++ b/kernel/hung_task.c
> > @@ -223,8 +223,11 @@ static inline void debug_show_blocker(struct task_struct *task, unsigned long ti
> >   }
> >   #endif
> > -static void check_hung_task(struct task_struct *t, unsigned long timeout)
> > +static void check_hung_task(struct task_struct *t, unsigned long timeout,
> > +		unsigned long prev_detect_count)
> >   {
> > +	unsigned long total_hung_task;
> > +
> >   	if (!task_is_hung(t, timeout))
> >   		return;
> > @@ -234,13 +237,19 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout)
> >   	 */
> >   	sysctl_hung_task_detect_count++;
> > +	total_hung_task = sysctl_hung_task_detect_count - prev_detect_count;
> >   	trace_sched_process_hang(t);
> > +	if (sysctl_hung_task_panic && total_hung_task >= sysctl_hung_task_panic) {
> > +		console_verbose();
> > +		hung_task_call_panic = true;
> > +	}
> > +
> >   	/*
> >   	 * Ok, the task did not get scheduled for more than 2 minutes,
> >   	 * complain:
> >   	 */
> > -	if (sysctl_hung_task_warnings) {
> > +	if (sysctl_hung_task_warnings || hung_task_call_panic) {
> >   		if (sysctl_hung_task_warnings > 0)
> >   			sysctl_hung_task_warnings--;
> >   		pr_err("INFO: task %s:%d blocked for more than %ld seconds.\n",
> > @@ -295,7 +304,6 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
> >   {
> >   	int max_count = sysctl_hung_task_check_count;
> >   	unsigned long last_break = jiffies;
> > -	unsigned long total_hung_task;
> >   	struct task_struct *g, *t;
> >   	unsigned long prev_detect_count = sysctl_hung_task_detect_count;
> >   	int need_warning = sysctl_hung_task_warnings;
> > @@ -320,20 +328,14 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
> >   			last_break = jiffies;
> >   		}
> > -		check_hung_task(t, timeout);
> > +		check_hung_task(t, timeout, prev_detect_count);
> >   	}
> >    unlock:
> >   	rcu_read_unlock();
> > -	total_hung_task = sysctl_hung_task_detect_count - prev_detect_count;
> > -	if (!total_hung_task)
> > +	if (!(sysctl_hung_task_detect_count - prev_detect_count))
> >   		return;
> > -	if (sysctl_hung_task_panic && total_hung_task >= sysctl_hung_task_panic) {
> > -		console_verbose();
> > -		hung_task_call_panic = true;
> > -	}
> > -
> >   	if (need_warning || hung_task_call_panic) {
> >   		si_mask |= SYS_INFO_LOCKS;
> 
> Looks good to me now! I assume v3 would be expected, can you
> post a new version?

Andrew has taken the patchset to -mm tree. 

Andrew, which way do you prefer? I send a v3 patch for hung-task or you
pickup the fixup patch and squash it into the orginal 0002 patch?

Anyway, I make a squshed version v3 patch below.

Thanks,
Feng

---
>From f90a60dae2440c89da7151fb1ddac022b872fb69 Mon Sep 17 00:00:00 2001
From: Feng Tang <feng.tang@...ux.alibaba.com>
Date: Wed, 5 Nov 2025 19:30:36 +0800
Subject: [PATCH v3] hung_task: Add hung_task_sys_info sysctl to dump sys info on task-hung

When task-hung happens, developers may need different kinds of system
information (call-stacks, memory info, locks, etc.) to help debugging.

Add 'hung_task_sys_info' sysctl knob to take human readable string like
"tasks,mem,timers,locks,ftrace,...", and when task-hung happens, all
requested information will be dumped. (refer kernel/sys_info.c for more
details).

Meanwhile, the newly introduced sys_info() call is used to unify some
existing info-dumping knobs.

Suggested-by: Petr Mladek <pmladek@...e.com>
Signed-off-by: Feng Tang <feng.tang@...ux.alibaba.com>
---
Changelog:

  v3:
  * restore hung_task_call_panic logic (Lance)

  v2:
  * code cleanup for si_mask setup (Petr)

 Documentation/admin-guide/sysctl/kernel.rst |  5 +++
 kernel/hung_task.c                          | 40 ++++++++++++++-------
 2 files changed, 33 insertions(+), 12 deletions(-)

diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst
index a397eeccaea7..45b4408dad31 100644
--- a/Documentation/admin-guide/sysctl/kernel.rst
+++ b/Documentation/admin-guide/sysctl/kernel.rst
@@ -422,6 +422,11 @@ the system boot.
 
 This file shows up if ``CONFIG_DETECT_HUNG_TASK`` is enabled.
 
+hung_task_sys_info
+==================
+A comma separated list of extra system information to be dumped when
+hung task is detected, for example, "tasks,mem,timers,locks,...".
+Refer 'panic_sys_info' section below for more details.
 
 hung_task_timeout_secs
 ======================
diff --git a/kernel/hung_task.c b/kernel/hung_task.c
index 5ac0e66a1361..d2254c91450b 100644
--- a/kernel/hung_task.c
+++ b/kernel/hung_task.c
@@ -24,6 +24,7 @@
 #include <linux/sched/sysctl.h>
 #include <linux/hung_task.h>
 #include <linux/rwsem.h>
+#include <linux/sys_info.h>
 
 #include <trace/events/sched.h>
 
@@ -59,12 +60,17 @@ static unsigned long __read_mostly sysctl_hung_task_check_interval_secs;
 static int __read_mostly sysctl_hung_task_warnings = 10;
 
 static int __read_mostly did_panic;
-static bool hung_task_show_lock;
 static bool hung_task_call_panic;
-static bool hung_task_show_all_bt;
 
 static struct task_struct *watchdog_task;
 
+/*
+ * A bitmask to control what kinds of system info to be printed when
+ * a hung task is detected, it could be task, memory, lock etc. Refer
+ * include/linux/sys_info.h for detailed bit definition.
+ */
+static unsigned long hung_task_si_mask;
+
 #ifdef CONFIG_SMP
 /*
  * Should we dump all CPUs backtraces in a hung task event?
@@ -236,7 +242,6 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout,
 
 	if (sysctl_hung_task_panic && total_hung_task >= sysctl_hung_task_panic) {
 		console_verbose();
-		hung_task_show_lock = true;
 		hung_task_call_panic = true;
 	}
 
@@ -259,10 +264,7 @@ static void check_hung_task(struct task_struct *t, unsigned long timeout,
 			" disables this message.\n");
 		sched_show_task(t);
 		debug_show_blocker(t, timeout);
-		hung_task_show_lock = true;
 
-		if (sysctl_hung_task_all_cpu_backtrace)
-			hung_task_show_all_bt = true;
 		if (!sysctl_hung_task_warnings)
 			pr_info("Future hung task reports are suppressed, see sysctl kernel.hung_task_warnings\n");
 	}
@@ -304,6 +306,8 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	unsigned long last_break = jiffies;
 	struct task_struct *g, *t;
 	unsigned long prev_detect_count = sysctl_hung_task_detect_count;
+	int need_warning = sysctl_hung_task_warnings;
+	unsigned long si_mask = hung_task_si_mask;
 
 	/*
 	 * If the system crashed already then all bets are off,
@@ -312,7 +316,7 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	if (test_taint(TAINT_DIE) || did_panic)
 		return;
 
-	hung_task_show_lock = false;
+
 	rcu_read_lock();
 	for_each_process_thread(g, t) {
 
@@ -328,14 +332,19 @@ static void check_hung_uninterruptible_tasks(unsigned long timeout)
 	}
  unlock:
 	rcu_read_unlock();
-	if (hung_task_show_lock)
-		debug_show_all_locks();
 
-	if (hung_task_show_all_bt) {
-		hung_task_show_all_bt = false;
-		trigger_all_cpu_backtrace();
+	if (!(sysctl_hung_task_detect_count - prev_detect_count))
+		return;
+
+	if (need_warning || hung_task_call_panic) {
+		si_mask |= SYS_INFO_LOCKS;
+
+		if (sysctl_hung_task_all_cpu_backtrace)
+			si_mask |= SYS_INFO_ALL_BT;
 	}
 
+	sys_info(si_mask);
+
 	if (hung_task_call_panic)
 		panic("hung_task: blocked tasks");
 }
@@ -434,6 +443,13 @@ static const struct ctl_table hung_task_sysctls[] = {
 		.mode		= 0444,
 		.proc_handler	= proc_doulongvec_minmax,
 	},
+	{
+		.procname	= "hung_task_sys_info",
+		.data		= &hung_task_si_mask,
+		.maxlen         = sizeof(hung_task_si_mask),
+		.mode		= 0644,
+		.proc_handler	= sysctl_sys_info_handler,
+	},
 };
 
 static void __init hung_task_sysctl_init(void)
-- 
2.43.5

Powered by blists - more mailing lists