linux-kernel - RE: Linux 6.10-rc2 - massive performance regression

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <adbbd899aabf4e6898bbbb04f90b3ede@AcuMS.aculab.com>
Date: Sat, 8 Jun 2024 21:35:48 +0000
From: David Laight <David.Laight@...LAB.COM>
To: 'Linus Torvalds' <torvalds@...ux-foundation.org>
CC: Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: RE: Linux 6.10-rc2 - massive performance regression

From: Linus Torvalds
> Sent: 08 June 2024 17:03
> 
> On Sat, 8 Jun 2024 at 07:44, David Laight <David.Laight@...lab.com> wrote:
> >
> > I suspect it is due to the changes to the way the scheduler pre-empts
> > processes.
> 
> The PREEMPT_AUTO changes haven't even made it into the kernel yet,
> only a couple of preliminary rcu patches that aren't relevant yet.
> 
> So while there's been some scheduler changes in 6.10, they have mostly
> been pretty insignificant - mostly some renaming without functional
> changes, and one-liners.
> 
> Obviously even a one-liner can end up resulting in random oddities,
> but could you try to bisect the issue you see to pin it down?

I'll try to remember how to bisect through the merge :-)

I've done some tests.
I'm seeing a three-fold slow down on:
$ i=0; while [ $i -lt 1000000 ]; do i=$((i + 1)); done
which goes from 1 second to 3.

I can run that with ftrace monitoring scheduler events (and a few
other things) and can't spot anywhere the process isn't running
for a significant time.

I've just had a thought.
Have there been any changes to the code that changes the cpu speed
and/or scheduler decisions for migrating processes?

It might be that the busy single process keeps on being migrated
to an idle cpu that is running at 800MHz rather than staying on
a cpu and getting its speed ramped up to 3.5-4GHz

I can speed the shell loop up to just over 2 seconds by running
a few copies of:
$ while :; do :; done
That might be the best I can get since I'm using an i7-7700 (4 cores
with hyperthreading giving 8 cpu) and hyperthreading probably slows
things down by a factor of two.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)