linux-kernel - 5.4.188 and later: massive performance regression with nfsd

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 11 May 2022 12:03:13 +0200
From:   Wolfgang Walter <linux@...m.de>
To:     stable@...r.kernel.org, Trond Myklebust <trondmy@...il.com>,
        Chuck Lever <chuck.lever@...cle.com>,
        linux-nfs@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: 5.4.188 and later: massive performance regression with nfsd

Hi,

starting with 5.4.188 wie see a massive performance regression on our 
nfs-server. It basically is serving requests very very slowly with cpu 
utilization of 100% (with 5.4.187 and earlier it is 10%) so that it is 
unusable as a fileserver.

The culprit are commits (or one of it):

c32f1041382a88b17da5736886da4a492353a1bb "nfsd: cleanup 
nfsd_file_lru_dispose()"
628adfa21815f74c04724abc85847f24b5dd1645 "nfsd: Containerise filecache 
laundrette"

(upstream 36ebbdb96b694dd9c6b25ad98f2bbd263d022b63 and 
9542e6a643fc69d528dfb3303f145719c61d3050)

If I revert them in v5.4.192 the kernel works as before and performance 
is ok again.

I did not try to revert them one by one as any disruption of our 
nfs-server is a severe problem for us and I'm not sure if they are 
related.

5.10 and 5.15 both always performed very badly on our nfs-server in a 
similar way so we were stuck with 5.4.

I now think this is because of 36ebbdb96b694dd9c6b25ad98f2bbd263d022b63 
and/or 9542e6a643fc69d528dfb3303f145719c61d3050 though I didn't tried to 
revert them in 5.15 yet.

Regards,
-- 
Wolfgang Walter
Studentenwerk München
Anstalt des öffentlichen Rechts