linux-kernel - Re: [RFC PATCH v2 0/7] Defer throttle when task exits to user

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <e65a32af-271b-4de6-937a-1a1049bbf511@amd.com>
Date: Tue, 15 Apr 2025 21:19:20 +0530
From: K Prateek Nayak <kprateek.nayak@....com>
To: Jan Kiszka <jan.kiszka@...mens.com>, Aaron Lu <ziqianlu@...edance.com>,
	Florian Bezdeka <florian.bezdeka@...mens.com>
CC: Valentin Schneider <vschneid@...hat.com>, Ben Segall <bsegall@...gle.com>,
	Peter Zijlstra <peterz@...radead.org>, Josh Don <joshdon@...gle.com>, Ingo
 Molnar <mingo@...hat.com>, Vincent Guittot <vincent.guittot@...aro.org>, Xi
 Wang <xii@...gle.com>, <linux-kernel@...r.kernel.org>, Juri Lelli
	<juri.lelli@...hat.com>, Dietmar Eggemann <dietmar.eggemann@....com>, Steven
 Rostedt <rostedt@...dmis.org>, Mel Gorman <mgorman@...e.de>, Chengming Zhou
	<chengming.zhou@...ux.dev>, Chuyi Zhou <zhouchuyi@...edance.com>, "Sebastian
 Andrzej Siewior," <bigeasy@...utronix.de>
Subject: Re: [RFC PATCH v2 0/7] Defer throttle when task exits to user

Hello Jan,

Sorry for the noise.

On 4/15/2025 4:46 PM, K Prateek Nayak wrote:
> Hello Jan,
> 
> On 4/15/2025 3:51 PM, Jan Kiszka wrote:
>>> Is this in line with what you are seeing?
>>>
>>
>> Yes, and if you wait a bit longer for the second reporting round, you
>> should get more task backtraces as well.
> 
> So looking at the backtrace [1], Aaron's patch should help with the
> stalls you are seeing.
> 
> timerfd that queues a hrtimer also uses ep_poll_callback() to wakeup
> the epoll waiter which queues ahead of the bandwidth timer and
> requires the read lock but now since the writer tried to grab the
> lock pushing readers on the slowpath. if epoll-stall-writer is now
> throttled, it needs ktimer to replenish its bandwidth which cannot
> happen without it grabbing the read lock first.
> 
> # epoll-stall-writer

So I got confused between "epoll-stall" and "epoll-stall-writer" here.
Turns out the actual series of events (based on traces, and hopefully
correct this time) are slightly longer. The correct series of events
are:

# epoll-stall-writer

anon_pipe_write()
   __wake_up_common()
     ep_poll_callback() {
       read_lock_irq(&ep->lock)		/* Read lock acquired here */
       __wake_up_common()
         ep_autoremove_wake_function()
           try_to_wake_up()		/* Wakes up "epoll-stall" */
             preempt_schedule()
             ...

# "epoll-stall-writer" has run out of bandwidth, needs replenish to run
# sched_switch: "epoll-stall-writer" => "epoll-stall"

     ... /* Resumes from epoll_wait() */
     epoll_wait() => 1			/* Write to FIFO */
     read() 				/* Reads one byte of data */
     epoll_wait()
       write_lock_irq()			/* Tries to grab write lock; "epoll-stall-writer" still has read lock */
         schedule_rtlock()		/* Sleeps but put next readers on slowpath */
         ...

# sched_switch: "epoll-stall" => "swapper"
# CPU is idle

...

# Timer interrupt schedules ktimers
# sched_switch: "swapper" => "ktimers"

hrtimer_run_softirq()
   timerfd_tmrproc()
     __wake_up_common()
       ep_poll_callback() {
         read_lock_irq(&ep->lock)	/* Blocks since we are in rwlock slowpath */
           schedule_rtlock()
           ...

# sched_switch: "ktimers" => "swapper"
# Bandwidth replenish never happens
# Stall

 From a second look at trace, this should be the right series of
events since "epoll-stall-writer" with bandwidth control seems
to have cut off during while doing the wakeup and hasn't run
again.

Sorry for the noise.

[..snip..]

> 
> [1] https://lore.kernel.org/all/62304351-7fc0-48b6-883b-d346886dac8e@amd.com/
> 
>>
>> Jan
>>
> 

-- 
Thanks and Regards,
Prateek