linux-kernel - crash on s390 with SCHED

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <yt9d35qwa82w.fsf@linux.ibm.com>
Date:   Thu, 26 Aug 2021 11:06:31 +0200
From:   Sven Schnelle <svens@...ux.ibm.com>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     linux-kernel@...r.kernel.org, linux-s390@...r.kernel.org,
        hca@...ux.ibm.com
Subject: crash on s390 with SCHED_CORE=y

Hi Peter,

i'm seeing the following crash on s390 with the strace testsuite (namely
the prctl-sched-core--pidns-translation testcase).

[   25.044039] Unable to handle kernel pointer dereference in virtual kernel address space
[   25.044051] Failing address: 0000000000000000 TEID: 0000000000000483
[   25.044053] Fault in home space mode while using kernel ASCE.
[   25.044056] AS:0000000166bd4007 R3:00000003fffe8007 S:00000003fffed800 P:000000000000003d
[   25.044080] Oops: 0004 ilc:2 [#1] SMP
[   25.044085] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink sunrpc dm_multipath scsi_dh_rdac scsi_dh_emc scsi_dh_alua s390_trng ghash_s390 ism p
rng smc aes_s390 des_s390 libdes ib_core sha3_512_s390 sha3_256_s390 sha512_s390 sha256_s390 sha1_s390 sha_common eadm_sch vfio_ccw mdev vfio_iommu_type1 vfio sch_fq_codel pkey zcrypt rng_core configfs ip_tables x_tables autofs4
[   25.044125] CPU: 122 PID: 2045 Comm: prctl-sched-cor Not tainted 5.14.0-rc7-08025-gf6d7568b37df-dirty #532
[   25.044129] Hardware name: IBM 8561 T01 701 (LPAR)
[   25.044130] Krnl PSW : 0404e00180000000 00000001655b7534 (raw_spin_rq_lock_nested+0x5c/0xb8)
[   25.044142]            R:0 T:1 IO:0 EX:0 Key:0 M:1 W:0 P:0 AS:3 CC:2 PM:0 RI:0 EA:3
[   25.044145] Krnl GPRS: ffffffffffffffff 0000000000000000 000000030000007b 0000000000000000
[   25.044147]            0000000000000000 0000000000000000 0000000100000080 0000000166ad0900
[   25.044149]            0000000000000400 0000000100000000 0000000000000000 00000003fd9a3600
[   25.044150]            000000009fb50000 0000000166172538 00000380077cbd00 00000380077cbcc0
[   25.044158] Krnl Code: 00000001655b752a: b90400ab            lgr     %r10,%r11
[   25.044158]            00000001655b752e: 582003ac            l       %r2,940
[   25.044158]           #00000001655b7532: 1819                lr      %r1,%r9
[   25.044158]           >00000001655b7534: ba12a000            cs      %r1,%r2,0(%r10)
[   25.044158]            00000001655b7538: ec16001d007e        cij     %r1,0,6,00000001655b7572
[   25.044158]            00000001655b753e: e310b1300112        lt      %r1,4400(%r11)
[   25.044158]            00000001655b7544: a7740012            brc     7,00000001655b7568
[   25.044158]            00000001655b7548: b904001b            lgr     %r1,%r11
[   25.044230] Call Trace:
[   25.044234]  [<00000001655b7534>] raw_spin_rq_lock_nested+0x5c/0xb8
[   25.044241]  [<00000001655d2cfc>] online_fair_sched_group+0x9c/0x1c0
[   25.044248]  [<00000001655e481c>] sched_autogroup_create_attach+0xdc/0x210
[   25.044253]  [<000000016559a45e>] ksys_setsid+0x10e/0x148
[   25.044258]  [<000000016559a4ba>] __s390_sys_setsid+0x22/0x30
[   25.044261]  [<000000016608402c>] __do_syscall+0x1bc/0x1e8
[   25.044266]  [<0000000166091358>] system_call+0x78/0xa0
[   25.044269] Last Breaking-Event-Address:
[   25.044270]  [<0000000000000000>] 0x0
[   25.044274] Kernel panic - not syncing: Fatal exception: panic_on_oops

The crash is caused by using a NULL rq->core value in
__rq_lockp, which is then passed on to raw_spin_rq_lock_nested(). This
is because it is implemented as follows:

static inline raw_spinlock_t *__rq_lockp(struct rq *rq)
{
        if (rq->core_enabled)
		return &rq->core->__lock;
	return &rq->__lock;
}

The problem is, that __sched_core_flip() sets rq->core_enabled to 1 even
for offline CPUS:

	/*
	 * Toggle the offline CPUs.
	 */
	cpumask_copy(&sched_core_mask, cpu_possible_mask);
	cpumask_andnot(&sched_core_mask, &sched_core_mask, cpu_online_mask);

	for_each_cpu(cpu, &sched_core_mask)
		cpu_rq(cpu)->core_enabled = enabled;

... But rq->core seems to get only assigned in the cpu hotplug notifer
sched_core_cpu_starting(), which is never called for offline CPUs. At
least i haven't found another place which assigns rq->core.

Any thoughts?

Thanks
Svem