linux-kernel - Re: 5.6-rc3: WARNING: CPU: 48 PID: 17435 at kernel/sched/fair.c:380 enqueue_task

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <f28bc5ac-87fa-2494-29db-ff7d98b7372a@de.ibm.com>
Date:   Wed, 4 Mar 2020 20:59:33 +0100
From:   Christian Borntraeger <borntraeger@...ibm.com>
To:     Dietmar Eggemann <dietmar.eggemann@....com>,
        Vincent Guittot <vincent.guittot@...aro.org>
Cc:     Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: 5.6-rc3: WARNING: CPU: 48 PID: 17435 at kernel/sched/fair.c:380
 enqueue_task_fair+0x328/0x440


On 04.03.20 20:38, Christian Borntraeger wrote:
> 
> 
> On 04.03.20 20:19, Dietmar Eggemann wrote:
>>> I just realized that this system has something special. Some month ago I created 2 slices
>>> $ head /etc/systemd/system/*.slice
>>> ==> /etc/systemd/system/machine-production.slice <==
>>> [Unit]
>>> Description=VM production
>>> Before=slices.target
>>> Wants=machine.slice
>>> [Slice]
>>> CPUQuota=2000%
>>> CPUWeight=1000
>>>
>>> ==> /etc/systemd/system/machine-test.slice <==
>>> [Unit]
>>> Description=VM production
>>> Before=slices.target
>>> Wants=machine.slice
>>> [Slice]
>>> CPUQuota=300%
>>> CPUWeight=100
>>>
>>>
>>> And the guests are then put into these slices. that also means that this test will never use more than the 2300%.
>>> No matter how much CPUs the system has.
>>
>> If you could run this debug patch on top of your un-patched kernel, it would tell us which task (in the enqueue case)
>> and which taskgroup is causing that.
>>
>> You could then further dump the appropriate taskgroup directory under the cpu cgroup mountpoint
>> (to see e.g. the CFS bandwidth data). 
>>
>> I expect more than one hit since assert_list_leaf_cfs_rq() uses SCHED_WARN_ON, hence WARN_ONCE.
> 
> That was quick. FWIW, I messed up dumping the cgroup mountpoint (since I restarted my guests after this happened).
> Will retry. See the dmesg attached. 

New occurence (with just one extra debug line)




View attachment "output" of type "text/plain" (33632 bytes)

Download attachment "sysfs.tar.lz4" of type "application/x-lz4" (90058 bytes)