[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <xhsmh4jsmieqx.mognet@vschneid.remote.csb>
Date: Thu, 19 Jan 2023 18:03:02 +0000
From: Valentin Schneider <vschneid@...hat.com>
To: Wander Lairson Costa <wander@...hat.com>,
Ingo Molnar <mingo@...hat.com>,
Peter Zijlstra <peterz@...radead.org>,
Juri Lelli <juri.lelli@...hat.com>,
Vincent Guittot <vincent.guittot@...aro.org>,
Dietmar Eggemann <dietmar.eggemann@....com>,
Steven Rostedt <rostedt@...dmis.org>,
Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
Daniel Bristot de Oliveira <bristot@...hat.com>,
"open list:SCHEDULER" <linux-kernel@...r.kernel.org>
Cc: Wander Lairson Costa <wander@...hat.com>,
Paul McKenney <paulmck@...nel.org>,
Thomas Gleixner <tglx@...utronix.de>
Subject: Re: [PATCH] sched/deadline: fix inactive_task_timer splat with
CONFIG_PREEMPT_RT
On 04/01/23 15:17, Wander Lairson Costa wrote:
> inactive_task_timer() executes in interrupt (atomic) context. It calls
> put_task_struct(), which indirectly acquires sleeping locks under
> PREEMPT_RT.
>
> Below is an example of a splat that happened in a test environment:
>
> CPU: 1 PID: 2848 Comm: life Kdump: loaded Tainted: G W ---------
> Hardware name: HP ProLiant DL388p Gen8, BIOS P70 07/15/2012
> Call Trace:
> dump_stack_lvl+0x57/0x7d
> mark_lock_irq.cold+0x33/0xba
> ? stack_trace_save+0x4b/0x70
> ? save_trace+0x55/0x150
> mark_lock+0x1e7/0x400
> mark_usage+0x11d/0x140
> __lock_acquire+0x30d/0x930
> lock_acquire.part.0+0x9c/0x210
> ? refill_obj_stock+0x3d/0x3a0
> ? rcu_read_lock_sched_held+0x3f/0x70
> ? trace_lock_acquire+0x38/0x140
> ? lock_acquire+0x30/0x80
> ? refill_obj_stock+0x3d/0x3a0
> rt_spin_lock+0x27/0xe0
> ? refill_obj_stock+0x3d/0x3a0
> refill_obj_stock+0x3d/0x3a0
> ? inactive_task_timer+0x1ad/0x340
> kmem_cache_free+0x357/0x560
> inactive_task_timer+0x1ad/0x340
> ? switched_from_dl+0x2d0/0x2d0
> __run_hrtimer+0x8a/0x1a0
> __hrtimer_run_queues+0x91/0x130
> hrtimer_interrupt+0x10f/0x220
> __sysvec_apic_timer_interrupt+0x7b/0xd0
> sysvec_apic_timer_interrupt+0x4f/0xd0
> ? asm_sysvec_apic_timer_interrupt+0xa/0x20
> asm_sysvec_apic_timer_interrupt+0x12/0x20
> RIP: 0033:0x7fff196bf6f5
>
> Instead of calling put_task_struct() directly, we defer it using
> call_rcu(). A more natural approach would use a workqueue, but since
> in PREEMPT_RT, we can't allocate dynamic memory from atomic context,
> the code would become more complex because we would need to put the
> work_struct instance in the task_struct and initialize it when we
> allocate a new task_struct.
>
Sorry to come back on this; Juri reminded me offline that put_task_struct()
is invoked in other non-sleepable contexts, not just inactive_task_timer().
e.g.
rto_push_irq_work_func() // hard irq work so hardirq context
`\
push_rt_task()
`\
put_task_struct()
Or
cpu_stopper_thread() // stopper callbacks must not sleep
`\
push_cpu_stop()
`\
put_task_struct()
... But then again I'm not aware of any splats happening in these paths. Is
there something special about inactive_task_timer(), or could it be the
issue is there for those other paths but we just haven't had them reported
yet?
Powered by blists - more mailing lists