linux-kernel - Re: [4.9-rc1] Build-time 2x slower

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1987803.InoJn4cnZD@vostro.rjw.lan>
Date:   Thu, 20 Oct 2016 16:51:11 +0200
From:   "Rafael J. Wysocki" <rjw@...ysocki.net>
To:     sedat.dilek@...il.com
Cc:     Jörg Otte <jrg.otte@...il.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        "Rafael J. Wysocki" <rafael.j.wysocki@...el.com>,
        Michal Marek <mmarek@...e.com>,
        Ming Lei <tom.leiming@...il.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Linux PM list <linux-pm@...r.kernel.org>
Subject: Re: [4.9-rc1] Build-time 2x slower

On Thursday, October 20, 2016 04:38:39 PM Sedat Dilek wrote:
> On Thu, Oct 20, 2016 at 4:32 PM, Rafael J. Wysocki <rjw@...ysocki.net> wrote:
> > On Thursday, October 20, 2016 04:20:44 PM Sedat Dilek wrote:
> >> On Thu, Oct 20, 2016 at 1:15 PM, Rafael J. Wysocki <rjw@...ysocki.net> wrote:
> >> > On Thursday, October 20, 2016 09:41:34 AM Sedat Dilek wrote:
> >> >> On Wed, Oct 19, 2016 at 10:55 PM, Rafael J. Wysocki <rjw@...ysocki.net> wrote:
> >> >> > On Wednesday, October 19, 2016 06:59:35 PM Jörg Otte wrote:
> >> >> >> 2016-10-19 17:29 GMT+02:00 Linus Torvalds <torvalds@...ux-foundation.org>:
> >> >> >> > On Wed, Oct 19, 2016 at 4:07 AM, Jörg Otte <jrg.otte@...il.com> wrote:
> >> >> >> >>
> >> >> >> >> Additional info: I usally use schedutil governor.
> >> >> >> >> If I switch to performance governor problems go away.
> >> >> >> >> Maybe a cpufreq problem?
> >> >> >> >
> >> >> >> > Oh, I completely misread the original bug report, and then didn't read
> >> >> >> > your confirmation email right.
> >> >> >> >
> >> >> >> > I thought you had a slower build of the different kernels (when
> >> >> >> > building on the same kernel), and that the _build_ itself had slowed
> >> >> >> > down for some reason. But you're actually saying that doing the _same_
> >> >> >> > build actually takes longer when running on 4.9-rc1.
> >> >> >>
> >> >> >> Exactly!
> >> >> >>
> >> >> >> Btw: ondemand governor is also good.
> >> >> >>
> >> >> >> > There are a few small cpufreq changes there in between commit
> >> >> >> > 29fbff8698fc (that you reported was fine - please tell me I got _that_
> >> >> >> > right, at least?) and 4.9-rc1.
> >> >> >>
> >> >> >> Perfect! That's what I mean.
> >> >> >>
> >> >> >> > Adding Rafael to the cc.
> >> >> >> >
> >> >> >> > That said, none of them look all that likely to me. It *would* be good
> >> >> >> > if you could bisect it a bit (perhaps not fully, but a couple of
> >> >> >> > bisection steps to narrow down what area it is).
> >> >> >>
> >> >> >> I try that tomorrow.
> >> >> >
> >> >> > Well, please try commit ef98988ba369 (Merge tag 'pm-extra-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm) which is the
> >> >> > merge introducing the late cpufreq changes.  If the issue is there, please
> >> >> > try to revert commit 899bb6642f2a (cpufreq: skip invalid entries when searching
> >> >> > the frequency) which is the only cpufreq one that may matter for the schedutil
> >> >> > governor (and I have one fix for that commit queued up already).
> >> >> >
> >> >>
> >> >> Is "cpufreq: fix overflow in cpufreq_table_find_index_dl()" the fix
> >> >> you are speaking of?
> >> >>
> >> >> Fixes: 899bb6642f2a (cpufreq: skip invalid entries when searching the frequency)
> >> >
> >> > Yes.
> >> >
> >> >> If yes, can you add a hint in the commit message describing the impact
> >> >> like here a slow-down of building a linux-kernel.
> >> >> With a reference to this ML-thread?
> >> >
> >> > I will if that turns out to be the case.
> >> >
> >>
> >> I have tried the revert and the patch from Sergey Senozhatsk pending
> >> in linux-pm.git#linux-next.
> >> Both fixes the issue for me.
> >
> > OK, thanks for the confirmation!
> >
> >> Feel free to give appropriate credits and many thanks to Jörg.
> >>
> >> I tried 'make -j3' in my last build and it was approx. 5mins faster in
> >> my customized setup.
> >> Will switch back to 2 parallel-make-jobs - it's safer for me.
> >>
> >> Can you explain why this issue was not seen when building under Linux v4.8.x?
> >> [1] says...
> >> Cc: 4.8+ <stable@...r.kernel.org> # 4.8+
> >
> > The commit in question might not make it into 4.8.y yet.
> >
> 
> It's a bit terrifying to see these impacts of schedutil cpufreq-governor.
> 
> Do you have a test-case or how do you test with / for schedutil?

In the same way as for the other governors: run a bunch of workloads and
see if there's any impact on the execution times etc.

And of course schedutil was not the only thing affected in this case
quite obviously which you can see from the fix changelog.

Thanks,
Rafael