linux-kernel - 4.2: CONFIG_NO_HZ_FULL_ALL effectively disabling non-boot CPUs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 10 Oct 2015 22:14:25 +0300 (EEST)
From:	Meelis Roos <mroos@...ux.ee>
To:	Linux Kernel list <linux-kernel@...r.kernel.org>
cc:	Thomas Gleixner <tglx@...utronix.de>,
	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
	Frederic Weisbecker <fweisbec@...il.com>
Subject: 4.2: CONFIG_NO_HZ_FULL_ALL effectively disabling non-boot CPUs

Short summary: turning on CONFIG_NO_HZ_FULL_ALL seems to disable all 
non-boot CPUs for scheduler.

A couple of days ago I noticed that make -j8 on a 4-core i5 is very slow 
(with 4.3.0-rc4+git). Looking at top ('1' for per-cpu states), only 
first CPU is loaded and 3 other CPUs are 100% idle. This seems to be a 
problem on 3 of my desktop machines (different generation Intel: i5-660, 
i5-2400, i3-3220). All the computers run custom kernels.

Further investigation showed that CPU affinity was set to 1 (CPU0 only) 
for init and all the children. Kernel threads had affinities 1,2,4,8 
and f (seems normal).

Even more interesting was the behaviour after setting affinity to f for 
all userland processes and then running make -j4. The other cores were 
still idle!

Switching back to 4.2.0 with my config, the problem persisted. 4.2.3 as 
packaged by Debian worked fine. 4.0.0 and 4.1.0 with my config worked 
also fine. systemd and sysvinit behaved the same and no affinity was 
configured for systemd.

So did a kernel config bisection between my kernel config and Debian 
config and came to CONFIG_NO_HZ_FULL_ALL. Debian has it off, I had it 
on. Turning that off fixed the scheduling and the system spread the 
tasks to all the cores.

I do not remember changing this value for a long time, I set them after 
the settings were introduced and used it. So it seems it broken in 4.2.0 
but was working in 4.1 but I do not have 4.1 config saved anywhere 
(many make oldconfigs since).

Bisection between 4.1 and 4.2 is possible but not easy since the 
machines are usually actively used when I am near them.

-- 
Meelis Roos (mroos@...ux.ee)
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/