Message-ID: <CAK1f24=MV1=7HjuYrts0jJzLFCzS7p+sZuSwky1BarHOCgooEQ@mail.gmail.com>
Date: Mon, 23 Jun 2025 20:33:53 +0800
From: Lance Yang <ioworker0@...il.com>
To: akpm@...ux-foundation.org
Cc: zi.li@...ux.dev, anna.schumaker@...cle.com, boqun.feng@...il.com, 
	joel.granados@...nel.org, jstultz@...gle.com, kent.overstreet@...ux.dev, 
	leonylgao@...cent.com, linux-kernel@...r.kernel.org, longman@...hat.com, 
	mhiramat@...nel.org, mingo@...hat.com, mingzhe.yang@...com, 
	peterz@...radead.org, rostedt@...dmis.org, senozhatsky@...omium.org, 
	tfiga@...omium.org, will@...nel.org
Subject: Re: [PATCH RFC 0/3] extend hung task blocker tracking to rwsems

Gently ping ;p

On Thu, Jun 12, 2025 at 12:20 PM Lance Yang <ioworker0@...il.com> wrote:
>
> Hi all,
>
> Inspired by mutex blocker tracking[1], and having already extended it to
> semaphores, let's now add support for reader-writer semaphores (rwsems).
>
> The approach is simple: when a task enters TASK_UNINTERRUPTIBLE while
> waiting for an rwsem, we just call hung_task_set_blocker(). The hung task
> detector can then query the rwsem's owner to identify the lock holder.
>
> Tracking works reliably for writers, as there can only be a single writer
> holding the lock, and its task struct is stored in the owner field.
>
> The main challenge lies with readers. The owner field points to only one
> of many concurrent readers, so we might lose track of the blocker if that
> specific reader unlocks, even while others remain. This is not a
> significant issue, however. In practice, long-lasting lock contention is
> almost always caused by a writer. Therefore, reliably tracking the writer
> is the primary goal of this patch series ;)
>
> With this change, the hung task detector can now show blocker task's info
> like below:
>
> [Thu Jun 12 11:01:33 2025] INFO: task rw_sem_thread2:36526 blocked for more than 122 seconds.
> [Thu Jun 12 11:01:33 2025]       Tainted: G S         O        6.16.0-rc1 #1
> [Thu Jun 12 11:01:33 2025] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [Thu Jun 12 11:01:33 2025] task:rw_sem_thread2  state:D stack:0     pid:36526 tgid:36526 ppid:2      task_flags:0x208040 flags:0x00004000
> [Thu Jun 12 11:01:33 2025] Call Trace:
> [Thu Jun 12 11:01:33 2025]  <TASK>
> [Thu Jun 12 11:01:33 2025]  __schedule+0x7c7/0x1930
> [Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? _raw_spin_lock_irq+0x8a/0xe0
> [Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  schedule+0x6a/0x180
> [Thu Jun 12 11:01:33 2025]  schedule_preempt_disabled+0x15/0x30
> [Thu Jun 12 11:01:33 2025]  rwsem_down_write_slowpath+0x447/0x1090
> [Thu Jun 12 11:01:33 2025]  ? __pfx_rwsem_down_write_slowpath+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx___might_resched+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx_thread2_func+0x10/0x10 [rw_sem_test_2]
> [Thu Jun 12 11:01:33 2025]  down_write+0x125/0x140
> [Thu Jun 12 11:01:33 2025]  ? __pfx_down_write+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? msleep+0x91/0xf0
> [Thu Jun 12 11:01:33 2025]  ? __raw_spin_lock_irqsave+0x8c/0xf0
> [Thu Jun 12 11:01:33 2025]  thread2_func+0x37/0x70 [rw_sem_test_2]
> [Thu Jun 12 11:01:33 2025]  kthread+0x39f/0x750
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ret_from_fork+0x25d/0x320
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ret_from_fork_asm+0x1a/0x30
> [Thu Jun 12 11:01:33 2025]  </TASK>
> [Thu Jun 12 11:01:33 2025] INFO: task rw_sem_thread2:36526 <writer> blocked on an rw-semaphore likely owned by task rw_sem_thread1:36525 <writer>
> [Thu Jun 12 11:01:33 2025] task:rw_sem_thread1  state:S stack:0     pid:36525 tgid:36525 ppid:2      task_flags:0x208040 flags:0x00004000
> [Thu Jun 12 11:01:33 2025] Call Trace:
> [Thu Jun 12 11:01:33 2025]  <TASK>
> [Thu Jun 12 11:01:33 2025]  __schedule+0x7c7/0x1930
> [Thu Jun 12 11:01:33 2025]  ? __pfx___schedule+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __mod_timer+0x304/0xa80
> [Thu Jun 12 11:01:33 2025]  ? irq_work_queue+0x6a/0xa0
> [Thu Jun 12 11:01:33 2025]  ? __pfx_vprintk_emit+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  schedule+0x6a/0x180
> [Thu Jun 12 11:01:33 2025]  schedule_timeout+0xfb/0x230
> [Thu Jun 12 11:01:33 2025]  ? __pfx_schedule_timeout+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx_process_timeout+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? down_write+0xc4/0x140
> [Thu Jun 12 11:01:33 2025]  msleep_interruptible+0xbe/0x150
> [Thu Jun 12 11:01:33 2025]  ? __pfx_thread1_func+0x10/0x10 [rw_sem_test_2]
> [Thu Jun 12 11:01:33 2025]  thread1_func+0x37/0x60 [rw_sem_test_2]
> [Thu Jun 12 11:01:33 2025]  kthread+0x39f/0x750
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx__raw_spin_lock_irq+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ret_from_fork+0x25d/0x320
> [Thu Jun 12 11:01:33 2025]  ? __pfx_kthread+0x10/0x10
> [Thu Jun 12 11:01:33 2025]  ret_from_fork_asm+0x1a/0x30
> [Thu Jun 12 11:01:33 2025]  </TASK>
>
> [1] https://lore.kernel.org/all/174046694331.2194069.15472952050240807469.stgit@mhiramat.tok.corp.google.com/
>
> Thanks,
> Lance
>
> Lance Yang (3):
>   locking/rwsem: make owner helpers globally available
>   locking/rwsem: clear reader-owner on unlock to reduce false positives
>   hung_task: extend hung task blocker tracking to rwsems
>
>  include/linux/hung_task.h | 18 +++++++++---------
>  include/linux/rwsem.h     | 12 ++++++++++++
>  kernel/hung_task.c        | 29 +++++++++++++++++++++++++----
>  kernel/locking/rwsem.c    | 31 +++++++++++++++++++++++--------
>  4 files changed, 69 insertions(+), 21 deletions(-)
>
> --
> 2.49.0
>

Powered by blists - more mailing lists