linux-kernel - Re: [Question] Do we need remote charging for cpu and cpuacct subsys?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20210702201920.i4pacu2x76e6crbf@oracle.com>
Date:   Fri, 2 Jul 2021 16:19:20 -0400
From:   Daniel Jordan <daniel.m.jordan@...cle.com>
To:     Hao Lee <haolee.swjtu@...il.com>,
        Pavan Kondeti <pkondeti@...eaurora.org>,
        Wei Wang <wvw@...gle.com>
Cc:     linux-mm@...ck.org, cgroups@...r.kernel.org,
        linux-kernel@...r.kernel.org, tj@...nel.org
Subject: Re: [Question] Do we need remote charging for cpu and cpuacct subsys?

+ Android folks

On Fri, Jul 02, 2021 at 04:07:42PM -0400, Daniel Jordan wrote:
> Hello,
> 
> On Fri, Jul 02, 2021 at 08:26:27AM -0000, Hao Lee wrote:
> > memcg currently has a remote charging mechanism that can charge usage to other
> > memcg instead of the one the task belongs to.
> > 
> > In our environment, we need to account the cpu usage consumed by some kworkers
> > to a specific cgroup. Thus, we want to introduce a remote-charging mechanism to
> > cpu and cpuacct subsys in our kernel.
> 
> I also want to see this upstream, and am actually working on it right
> now, have been for some time.
> 
> So far, this is needed to properly account multithreaded padata jobs,
> memory reclaim, and net rx.  Android folks have raised this issue in the
> past too, though I'm not aware of the specific kthreads that are giving
> them problems.

Pavan, Wei, do you have any details about this?

> So naturally, I'm curious about your use case and how it may be
> different from these others.  What kworkers would you like to account?
> 
> > I want to know if the community has a plan to do this?
> > What will the community approach look like?
> 
> There has been discussion about this here,
> 
>    https://lore.kernel.org/lkml/20200219214112.4kt573kyzbvmbvn3@ca-dmjordan1.us.oracle.com/
> 
> more recently here,
> 
>    https://lore.kernel.org/lkml/YGxjwKbec68sCcqo@slm.duckdns.org/
> 
> and we may talk about it at LPC:
> 
>    https://www.linuxplumbersconf.org/event/11/page/104-accepted-microconferences#cont-perform
> 
> > I think we need to move the active_memcg to a separated active_cgroup struct,
> > and the latter will contain active_memcg, active_tg, and active_cpuacct.
> 
> I'm not seeing how that could work for cases that don't know the cgroup
> when the remote charging period begins.  The only one I'm aware of
> that's like that is net rx, where the work to process packets has to
> start before their ultimate destination, and therefore cgroup, is known.
> 
> thanks,
> Daniel