linux-kernel - Re: sched : performance regression 24% between 4.4rc4 and 4.3 kernel

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAE4VaGCMK1_aZSsGQ8xzSuooqZ4WGkciX62JQ=+4htdca9Ar=A@mail.gmail.com>
Date:	Wed, 16 Dec 2015 13:56:17 +0100
From:	Jirka Hladky <jhladky@...hat.com>
To:	Rik van Riel <riel@...hat.com>
Cc:	Mike Galbraith <umgwanakikbuti@...il.com>,
	linux-kernel@...r.kernel.org,
	Peter Zijlstra <peterz@...radead.org>,
	Ingo Molnar <mingo@...e.hu>,
	"kkolakow@...hat.com" <kkolakow@...hat.com>
Subject: Re: sched : performance regression 24% between 4.4rc4 and 4.3 kernel

Hi Rik,

I have redone the bisecting and have new results:

# first bad commit: [2a595721a1fa6b684c1c818f379bef834ac3d65e]
sched/numa: Convert sched_numa_balancing to a static_branch

Could you please have a look what went wrong?

Thanks a lot!
Jirka

git bisect start '--' 'kernel/sched'
# good: [6a13feb9c82803e2b815eca72fa7a9f5561d7861] Linux 4.3
git bisect good 6a13feb9c82803e2b815eca72fa7a9f5561d7861
# bad: [527e9316f8ec44bd53d90fb9f611fa7ffff52bb9] Linux 4.4-rc4
git bisect bad 527e9316f8ec44bd53d90fb9f611fa7ffff52bb9
# bad: [b99def8b961448f5b9a550dddeeb718e3975e7a6] sched/core: Rework
TASK_DEAD preemption exception
git bisect bad b99def8b961448f5b9a550dddeeb718e3975e7a6
# skip: [8cd5601c50603caa195ce86cc465cb04079ed488] sched/fair: Convert
arch_scale_cpu_capacity() from weak function to #define
git bisect skip 8cd5601c50603caa195ce86cc465cb04079ed488
# bad: [fe19159225d8516f3f57a5fe8f735c01684f0ddd] Merge branch
'sched/urgent' into sched/core, to pick up fixes before applying new
changes
git bisect bad fe19159225d8516f3f57a5fe8f735c01684f0ddd
# good: [78a9c54649ea220065aad9902460a1d137c7eafd] sched/numa: Rename
numabalancing_enabled to sched_numa_balancing
git bisect good 78a9c54649ea220065aad9902460a1d137c7eafd
# bad: [54a21385facbdcd89a78e8c3e5025f04c5f2b59c] sched/fair: Rename
scale() to cap_scale()
git bisect bad 54a21385facbdcd89a78e8c3e5025f04c5f2b59c
# bad: [9e91d61d9b0ca8d865dbd59af8d0d5c5b68003e9] sched/fair: Name
utilization related data and functions consistently
git bisect bad 9e91d61d9b0ca8d865dbd59af8d0d5c5b68003e9
# bad: [2a595721a1fa6b684c1c818f379bef834ac3d65e] sched/numa: Convert
sched_numa_balancing to a static_branch
git bisect bad 2a595721a1fa6b684c1c818f379bef834ac3d65e
# good: [2b49d84b259fc18e131026e5d38e7855352f71b9] sched/numa: Remove
the NUMA sched_feature
git bisect good 2b49d84b259fc18e131026e5d38e7855352f71b9
# first bad commit: [2a595721a1fa6b684c1c818f379bef834ac3d65e]
sched/numa: Convert sched_numa_balancing to a static_branch

On Tue, Dec 15, 2015 at 9:49 AM, Jirka Hladky <jhladky@...hat.com> wrote:
> Hi Rik,
>
> I have reviewed the data and you are right. The trouble is that even
> with 4.3 kernel there is 20% change that results will be bad. I have
> repeated tests 100 times on 4.3 kernel over the night. In 20 cases I
> see that runtime went up from 12 seconds to 28 seconds due to the
> wrong NUMA placement. I will try to replay the bisect once again.
>
> Jirka
>
> On Tue, Dec 15, 2015 at 3:12 AM, Rik van Riel <riel@...hat.com> wrote:
>> On 12/14/2015 06:52 PM, Jirka Hladky wrote:
>>> Hi all,
>>>
>>> I have the results of bisecting:
>>>
>>> first bad commit: [973759c80db96ed4b4c5cb85ac7d48107f801371] Merge tag
>>> 'v4.3-rc1' into sched/core, to refresh the branch
>>>
>>> Could you please have a look at this commit why it has caused the
>>> performance regression when running 4 stream benchmarks in parallel on 4
>>> NUMA node server?
>>
>> That is a merge commit. It contains no actual code changes.
>>
>>> Please let me know if you need additional data. git bisect log is bellow.
>>
>> It looks like "git bisect" may have led you astray.
>>
>> I am not sure what debugging tool to use to figure out which
>> of the patches from some merged-in branch caused the issue,
>> but hopefully one of the people reading this email know a trick.
>>
>> --
>> All rights reversed
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/