linux-kernel - Re: [RT BUG] Stall caused by eventpoll, rwlocks and CFS bandwidth controller

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <905b3cca-e9ab-4844-bbe2-aa9a8c78e047@siemens.com>
Date: Wed, 9 Apr 2025 15:52:19 +0200
From: Jan Kiszka <jan.kiszka@...mens.com>
To: Sebastian Andrzej Siewior <bigeasy@...utronix.de>
Cc: Valentin Schneider <vschneid@...hat.com>, linux-rt-users@...r.kernel.org,
 linux-kernel@...r.kernel.org, kprateek.nayak@....com,
 Thomas Gleixner <tglx@...utronix.de>, Juri Lelli <juri.lelli@...hat.com>,
 Clark Williams <williams@...hat.com>,
 "Luis Claudio R. Goncalves" <lgoncalv@...hat.com>,
 Andreas Ziegler <ziegler.andreas@...mens.com>,
 Felix Moessbauer <felix.moessbauer@...mens.com>,
 Florian Bezdeka <florian.bezdeka@...mens.com>
Subject: Re: [RT BUG] Stall caused by eventpoll, rwlocks and CFS bandwidth
 controller

On 09.04.25 15:41, Jan Kiszka wrote:
> On 09.04.25 15:21, Sebastian Andrzej Siewior wrote:
>> On 2025-04-09 08:41:44 [+0200], Jan Kiszka wrote:
>>> We are hunting for quite some time sporadic lock-ups or RT systems, 
>>> first only in the field (sigh), now finally also in the lab. Those have 
>>> a fairly high overlap with what was described here. Our baselines so 
>>> far: 6.1-rt, Debian and vanilla. We are currently preparing experiments 
>>> with latest mainline.
>>>
>>> While this thread remained silent afterwards, we have found [1][2][3] as 
>>> apparently related. But this means we are still with this RT bug, even 
>>> in latest 6.15-rc1?
>>
>> Not sure the commits are related. The problem here is that RW locks are
>> not really real time friendly. Frederick had a simple fix to it
>> 	https://lore.kernel.org/all/20210825132754.GA895675@lothringen/
>>
>> but yeah. The alternative, which I didn't look into, would be to replace
>> the reader side with RCU so we would just have the writer lock. That
>> mean we need to RW lock because of performance…
>>
> 
> We know that epoll is not a good idea for RT programs. However, our
> problem is that already non-RT programs manage to lock up an RT-enabled
> system.

On second glance, Frederic's patch would probably also avoid the issue
we are seeing as it should bring PI to the CFS-throttled read-lock
holder (which is now a write-lock holder).

But given how old that proposal is, I assume the performance impact was
even for the RT kernel too much, wasn't it?

Jan

-- 
Siemens AG, Foundational Technologies
Linux Expert Center