linux-kernel - Why does the 2.6.37 scheduler hate RT tasks?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 1 Feb 2011 17:10:49 -0800
From:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To:	peterz@...radead.org
Cc:	linux-kernel@...r.kernel.org
Subject: Why does the 2.6.37 scheduler hate RT tasks?

Hello, Peter,

If I create an SCHED_FIFO-priority kthread in pre-2.6.37 kernels,
things work nicely.  If I try in in later kernels, __disable_runtime()
gets upset occasionally during CPU-hotplug calls when CPUs are inserted.
If I make the exact same kthreads run SCHED_NORMAL, no problem.

Unfortunately, I need an RT kthread so that I can do priority boosting.

My next thing to try is to demote my kthreads to SCHED_NORMAL at the
beginning of a CPU-hotplug event and promote them back to SCHED_FIFO at
the end of the CPU-hotplug event, which would leave the system vulnerable
to RCU priority inversion during CPU-hotplug events.  This would be
better than the current behavior, but I thought that I should check with
you to see if there is a better approach.

Help?

							Thanx, Paul

PS.  The code at kernel/sched_rt.c:466 is "BUG_ON(want);" in
     __disable_runtime().  Config attached in case that helps.

------------------------------------------------------------------------

kernel BUG at kernel/sched_rt.c:466!
Oops: Exception in kernel mode, sig: 5 [#1]
SMP NR_CPUS=8 NUMA pSeries
last sysfs file: /sys/devices/system/cpu/cpu2/online
Modules linked in: rcutorture [last unloaded: scsi_wait_scan]
NIP: c0000000000727d4 LR: c0000000000727d0 CTR: c000000000078620
REGS: c0000000f8da3070 TRAP: 0700   Not tainted  (2.6.37-autokern1)
MSR: 8000000000029032 <EE,ME,CE,IR,DR>  CR: 24248422  XER: 20000000
TASK = c0000000f825c1f0[19711] 'sh' THREAD: c0000000f8da0000 CPU: 1
GPR00: c0000000000727d0 c0000000f8da32f0 c000000000eae7b0 c0000000013e6e78 
GPR04: 0000000000000008 0000000000000008 ffffffffffffffff 0000000000000001 
GPR08: 0000000000000000 0000000080000001 ffffffffffffffff 00000000000000ff 
GPR12: 0000000044242422 c00000000fff0280 0000000000000000 c00000000103f668 
GPR16: 0000000000000000 0000000000000008 c0000000013e67c8 0000000000000000 
GPR20: ffffffffffffffff c0000000f8da3360 c0000000013e6eb8 c0000000013e6e78 
GPR24: c00000000103ee80 0000000000000000 c0000000f8997e88 c000000000f11e80 
GPR28: c00000000103eed0 c0000000013e67c8 c000000000e36978 0000000005f5e100 
NIP [c0000000000727d4] .__disable_runtime+0x1b4/0x2a0
LR [c0000000000727d0] .__disable_runtime+0x1b0/0x2a0
Call Trace:
[c0000000f8da32f0] [c0000000000727d0] .__disable_runtime+0x1b0/0x2a0 (unreliable)
[c0000000f8da33f0] [c0000000000786ac] .rq_offline_rt+0x8c/0xc0
[c0000000f8da3470] [c00000000006b2e8] .set_rq_offline+0x68/0xe0
[c0000000f8da3500] [c00000000007ae58] .rq_attach_root+0x1f8/0x210
[c0000000f8da35a0] [c00000000007b7ec] .cpu_attach_domain+0x1ac/0x250
[c0000000f8da3670] [c00000000007cbf4] .partition_sched_domains+0x1d4/0x4b0
[c0000000f8da3790] [c0000000000e82e0] .cpuset_update_active_cpus+0x80/0xa0
[c0000000f8da3840] [c00000000007b044] .cpuset_cpu_active+0x44/0x60
[c0000000f8da38b0] [c00000000062e1fc] .notifier_call_chain+0x9c/0x100
[c0000000f8da3950] [c000000000089bc8] .__cpu_notify+0x38/0x80
[c0000000f8da39d0] [c000000000634cc4] ._cpu_up+0x10c/0x178
[c0000000f8da3a90] [c000000000634e68] .cpu_up+0x138/0x164
[c0000000f8da3b20] [c000000000513be8] .store_online+0xc8/0x120
[c0000000f8da3bc0] [c00000000039bf7c] .sysdev_store+0x3c/0x50
[c0000000f8da3c30] [c000000000232dac] .sysfs_write_file+0xec/0x1f0
[c0000000f8da3ce0] [c0000000001a871c] .vfs_write+0xec/0x1e0
[c0000000f8da3d80] [c0000000001a8938] .SyS_write+0x58/0xb0
[c0000000f8da3e30] [c000000000008564] syscall_exit+0x0/0x40
Instruction dump:
7fe0f850 f93206a8 880d0214 2f800000 40de0088 7c2004ac 2fbf0000 933206b0 
409eff6c 7ee3bb78 485b6885 60000000 <0b1f0000> fa9d06a8 880d0214 2f800000

View attachment "config.txt" of type "text/plain" (61286 bytes)