[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Sat, 10 Oct 2015 12:24:39 -0700
From: "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
To: Meelis Roos <mroos@...ux.ee>
Cc: Linux Kernel list <linux-kernel@...r.kernel.org>,
Thomas Gleixner <tglx@...utronix.de>,
Frederic Weisbecker <fweisbec@...il.com>
Subject: Re: 4.2: CONFIG_NO_HZ_FULL_ALL effectively disabling non-boot CPUs
On Sat, Oct 10, 2015 at 10:14:25PM +0300, Meelis Roos wrote:
> Short summary: turning on CONFIG_NO_HZ_FULL_ALL seems to disable all
> non-boot CPUs for scheduler.
>
> A couple of days ago I noticed that make -j8 on a 4-core i5 is very slow
> (with 4.3.0-rc4+git). Looking at top ('1' for per-cpu states), only
> first CPU is loaded and 3 other CPUs are 100% idle. This seems to be a
> problem on 3 of my desktop machines (different generation Intel: i5-660,
> i5-2400, i3-3220). All the computers run custom kernels.
>
> Further investigation showed that CPU affinity was set to 1 (CPU0 only)
> for init and all the children. Kernel threads had affinities 1,2,4,8
> and f (seems normal).
>
> Even more interesting was the behaviour after setting affinity to f for
> all userland processes and then running make -j4. The other cores were
> still idle!
>
> Switching back to 4.2.0 with my config, the problem persisted. 4.2.3 as
> packaged by Debian worked fine. 4.0.0 and 4.1.0 with my config worked
> also fine. systemd and sysvinit behaved the same and no affinity was
> configured for systemd.
>
> So did a kernel config bisection between my kernel config and Debian
> config and came to CONFIG_NO_HZ_FULL_ALL. Debian has it off, I had it
> on. Turning that off fixed the scheduling and the system spread the
> tasks to all the cores.
>
> I do not remember changing this value for a long time, I set them after
> the settings were introduced and used it. So it seems it broken in 4.2.0
> but was working in 4.1 but I do not have 4.1 config saved anywhere
> (many make oldconfigs since).
>
> Bisection between 4.1 and 4.2 is possible but not easy since the
> machines are usually actively used when I am near them.
This is expected and intended behavior. The whole point of
CONFIG_NO_HZ_FULL_ALL is to keep everything off of the non-boot CPUs
that is not explicitly placed there. Without CONFIG_NO_HZ_FULL_ALL,
you can use the nohz_full boot parameter to select exactly which
CPUs are to behave this way.
Thanx, Paul
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists