linux-kernel - Re: 6.10/regression/bisected - after f1d97e769152 I spotted increased execution time of the kswapd0 process and symptoms as if there is not enough memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CABXGCsP9tSwgR4dN-k97maqHB1KOtykakmHNz78SYbAuHydUTQ@mail.gmail.com>
Date: Tue, 2 Jul 2024 19:13:20 +0500
From: Mikhail Gavrilov <mikhail.v.gavrilov@...il.com>
To: Filipe Manana <fdmanana@...nel.org>
Cc: Linux List Kernel Mailing <linux-kernel@...r.kernel.org>, 
	Linux regressions mailing list <regressions@...ts.linux.dev>, Btrfs BTRFS <linux-btrfs@...r.kernel.org>, 
	dsterba@...e.com, josef@...icpanda.com
Subject: Re: 6.10/regression/bisected - after f1d97e769152 I spotted increased
 execution time of the kswapd0 process and symptoms as if there is not enough memory

On Mon, Jul 1, 2024 at 2:31 PM Filipe Manana <fdmanana@...nel.org> wrote:
>
> Try this:
>
> https://lore.kernel.org/linux-btrfs/cb12212b9c599817507f3978c9102767267625b2.1719825714.git.fdmanana@suse.com/
>
> That applies only to the "for-next", it will need conflict resolution
> for 6.10-rc, as noted in the commnets.
> For a version that cleanly applies to 6.10-rc:
>
> https://gist.githubusercontent.com/fdmanana/5262e608b3eecb9a3b2631f8dad49863/raw/1a82fe8eafbd5f6958dddf34d3c9648d7335018e/btrfs-don-t-loop-again-over-pinned-extent-maps-when-.patch

I tested this patch on top of v6.10-rc6

> Btw, besides the longer kswapd execution times, what else do you observe?
> Is it impacting performance of any applications?

I observe that the system freezes under load.
Demonstration: https://youtu.be/1-gUrnEi2aU
The GNOME shell stops responding, and even the clock in the GNOME
status bar stops updating seconds.
And this didn't happen when the v6.9 kernel was running. Second, I
spotted high CPU usage by process kswapd0 when freezes occurred.
Therefore, I decided to find the commit that led to high CPU
consumption by the kswapd0 process.
As we found out, this commit turned out to be 956a17d9d050.

> I think no matter what we do, it's likely that kswapd will take more
> time than before, because now there's extra work of going through
> extent maps and dropping them.
> We had to do it to prevent OOM situations because extent map creation
> was unbounded.

Unfortunately, the patch didn't improve anything.
kswapd0 still consumes 100% CPU under load.
And my system continues to freeze.

6.10.0-0.rc6.51.fc41.x86_64+debug with patch
up  1:00
root         269 13.1  0.0      0     0 ?        S    12:24   7:53 [kswapd0]
up  2:00
root         269 29.9  0.0      0     0 ?        R    12:24  36:02 [kswapd0]
up  3:00
root         269 37.8  0.0      0     0 ?        S    12:24  68:19 [kswapd0]
up  4:05
root         269 39.3  0.0      0     0 ?        R    12:24  96:40 [kswapd0]
up  5:01
root         269 38.8  0.0      0     0 ?        R    12:24 117:00 [kswapd0]
up  6:00
root         269 40.3  0.0      0     0 ?        S    12:24 145:24 [kswapd0]

-- 
Best Regards,
Mike Gavrilov.