linux-kernel - Re: [PATCH] cgroup: rstat: use LOCK CMPXCHG in css_rstat

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aTMX4tycdzKlaqaH@slm.duckdns.org>
Date: Fri, 5 Dec 2025 07:35:30 -1000
From: Tejun Heo <tj@...nel.org>
To: Shakeel Butt <shakeel.butt@...ux.dev>
Cc: Johannes Weiner <hannes@...xchg.org>,
	Michal Koutný <mkoutny@...e.com>,
	"Paul E . McKenney" <paulmck@...nel.org>,
	JP Kobryn <inwardvessel@...il.com>,
	Yosry Ahmed <yosry.ahmed@...ux.dev>, linux-mm@...ck.org,
	cgroups@...r.kernel.org, linux-kernel@...r.kernel.org,
	Meta kernel team <kernel-team@...a.com>
Subject: Re: [PATCH] cgroup: rstat: use LOCK CMPXCHG in css_rstat_updated

Hello,

On Thu, Dec 04, 2025 at 06:24:37PM -0800, Shakeel Butt wrote:
...
> In Meta's fleet running the kernel with the commit 36df6e3dbd7e, we are
> observing on some machines the memcg stats are getting skewed by more
> than the actual memory on the system. On close inspection, we noticed
> that lockless node for a workload for specific CPU was in the bad state
> and thus all the updates on that CPU for that cgroup was being lost. At
> the moment, we are not sure if this CMPXCHG without LOCK is the cause of
> that but this needs to be fixed irrespective.

Is there a plausible theory of events that can explain the skew with the use
of this_cpu_cmpxchg()? lnode.next being set to self but this_cpu_cmpxchg()
returning something else? It may be useful to write a targeted repro for the
particular combination - this_cpu_cmpxchg() vs. remote NULL clearing and see
whether this_cpu_cmpxchg() can return a value that doesn't agree with what
gets written in the memory.

> @@ -113,9 +112,8 @@ __bpf_kfunc void css_rstat_updated(struct cgroup_subsys_state *css, int cpu)
>  	 * successful and the winner will eventually add the per-cpu lnode to
>  	 * the llist.
>  	 */
> -	self = &rstatc->lnode;
> -	rstatc_pcpu = css->rstat_cpu;
> -	if (this_cpu_cmpxchg(rstatc_pcpu->lnode.next, self, NULL) != self)
> +	expected = &rstatc->lnode;
> +	if (!try_cmpxchg(&rstatc->lnode.next, &expected, NULL))

Given that this is a relatively cold path, I don't see a problem with using
locked op here even if this wasn't necessarily the culprit; however, can you
please update the comment right above accordingly and explain why the locked
op is used? After this patch, the commend and code disagree.

Thanks.

-- 
tejun