linux-kernel - Re: [PATCH v3] memcg: fix soft lockup in the OOM process

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <362f274a-70ae-4588-afb3-e75ade66ea42@huaweicloud.com>
Date: Sat, 4 Jan 2025 08:52:35 +0800
From: Chen Ridong <chenridong@...weicloud.com>
To: Michal Koutný <mkoutny@...e.com>
Cc: akpm@...ux-foundation.org, mhocko@...nel.org, hannes@...xchg.org,
 yosryahmed@...gle.com, roman.gushchin@...ux.dev, shakeel.butt@...ux.dev,
 muchun.song@...ux.dev, davidf@...eo.com, vbabka@...e.cz,
 handai.szj@...bao.com, rientjes@...gle.com, kamezawa.hiroyu@...fujitsu.com,
 linux-mm@...ck.org, linux-kernel@...r.kernel.org, cgroups@...r.kernel.org,
 chenridong@...wei.com, wangweiyang2@...wei.com
Subject: Re: [PATCH v3] memcg: fix soft lockup in the OOM process



On 2025/1/4 0:18, Michal Koutný wrote:
> Hello.
> 
> On Tue, Dec 24, 2024 at 02:52:38AM +0000, Chen Ridong <chenridong@...weicloud.com> wrote:
>> A soft lockup issue was found in the product with about 56,000 tasks were
>> in the OOM cgroup, it was traversing them when the soft lockup was
>> triggered.
> 
> Why is this softlockup a problem? 
> It's lot of tasks afterall and possibly a slow console (given looking
> for a victim among the comparable number didn't trigger it).
> 

It's not a slow console, but rather 'console pressure'. When a lot of
tasks apply to the console, it can make 'pr_info' slow. In my case,
these tasks will apply to the console. I reproduced this issue using a
test ko that creates many tasks, all of which just call 'pr_info'.

Best regards,
Ridong

>> To fix this issue, call 'cond_resched' in the 'mem_cgroup_scan_tasks'
>> function per 1000 iterations. For global OOM, call
>> 'touch_softlockup_watchdog' per 1000 iterations to avoid this issue.
> 
> This only hides the issue. It could be similarly fixed by simply
> decreasing loglevel= ;-)
> 
> cond_resched() in the memcg case may be OK but the arbitrary touch for
> global situation may hide possibly useful troubleshooting information.
> (Yeah, cond_resched() won't fit inside RCU section as in other global
> task iterations.)
> 
> 0.02€,
> Michal