linux-kernel - Re: [PREEMPT-RT] Oops in rapl_cpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 21 Oct 2016 17:03:56 -0400
From:   "Charles (Chas) Williams" <ciwillia@...cade.com>
To:     Sebastian Andrzej Siewior <bigeasy@...utronix.de>
CC:     <linux-kernel@...r.kernel.org>, <rt@...utronix.de>
Subject: Re: [PREEMPT-RT] Oops in rapl_cpu_prepare()

On 10/21/2016 06:56 AM, Sebastian Andrzej Siewior wrote:
> On 2016-10-20 16:27:55 [-0400], Charles (Chas) Williams wrote:
>> Recent 4.8 kernels have been oopsing when running under VMWare:
>
> can you reproduce this on bare metal?

I can't get dedicated access to the specific bare metal since it is
running as a dedicated hypervisor.  I haven't seen this issue anywhere
else though with the 4.8 kernel.

>> [    2.270203] BUG: unable to handle kernel NULL pointer dereference at 0000000000000408
>> [    2.270325] IP: [<ffffffff81012bb9>] rapl_cpu_online+0x59/0x70
>
> can you check if pmu is NULL?

It's not.  The dereference at 0x408 and pmu->cpu being fairly early in
the struct seems to indicate that pmu wasn't pointing to 0 at the time
(but fairly close).  I should have noticed that earlier.

>> Is there a particular order guaranteed by the callbacks?  Will
>> rapl_cpu_prepare() always happen before online/offline?  Additionally,
>
> yes, see include/linux/cpuhotplug.h. On CPU-up the array ids are invoked
> from CPUHP_OFFLINE till CPUHP_ONLINE.

Yes, I see that now.  Thanks for the pointer!

> If a callback (such as CPUHP_PERF_X86_RAPL_PREP) fail then we rollback
> to the starting point (in case of CPU up it would be CPUHP_OFFLINE.

You'll like this, I just did a little printk debugging because it was
easier than trying to get a debugger running:

	[    3.107126] init_rapl_pmus: maxpkg 4
	[    3.107263] rapl_cpu_prepare: pmu ffff880234faa540  cpu 0  pkgid 0
	[    3.107400] rapl_cpu_prepare: pmu ffff880234faa600  cpu 1  pkgid 2
	[    3.107537] rapl_cpu_prepare: pmu ffff880234faa6c0  cpu 2  pkgid 65535
	[    3.107662] rapl_cpu_online: pmu ffff880234faa540 cpu 0 pkgid 0
	[    3.107907] rapl_cpu_online: pmu ffff880234faa600 cpu 1 pkgid 2
	[    3.108133] rapl_cpu_online: pmu ffff880234faa6c0 cpu 2 pkgid 65535
	[    3.108333] rapl_cpu_online: pmu ffff880234faa6c0 cpu 3 pkgid 65535

where pkgid is topology_logical_package_id(cpu).

I can't understand why I don't see a cpu 3 during cpu prepare, when I
see one later.  The 65535 is a -1 from topology_phys_to_logical_pkg()
getting assigned to the logical_proc_id apparently.

So this is pretty puzzling.  Since this is a guest running under VMWare, I
don't know that there is any particular CPU pinning or emulation of RAPL.

It looks there was a proposal to not run in guests:

https://lkml.org/lkml/2015/12/3/559