linux-kernel - Re: [RFC] Shared page accounting for memory cgroup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <661de9471001171130p2b0ac061he6f3dab9ef46fd06@mail.gmail.com>
Date:	Mon, 18 Jan 2010 01:00:44 +0530
From:	Balbir Singh <balbir@...ux.vnet.ibm.com>
To:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
Cc:	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"nishimura@....nes.nec.co.jp" <nishimura@....nes.nec.co.jp>
Subject: Re: [RFC] Shared page accounting for memory cgroup

On Fri, Jan 8, 2010 at 5:17 AM, KAMEZAWA Hiroyuki
<kamezawa.hiroyu@...fujitsu.com> wrote:
> On Thu, 7 Jan 2010 14:57:36 +0530
> Balbir Singh <balbir@...ux.vnet.ibm.com> wrote:
>
>> * KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com> [2010-01-07 18:08:00]:
>>
>> > On Thu, 7 Jan 2010 17:48:14 +0900
>> > KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com> wrote:
>> > > > > "How pages are shared" doesn't show good hints. I don't hear such parameter
>> > > > > is used in production's resource monitoring software.
>> > > > >
>> > > >
>> > > > You mean "How many pages are shared" are not good hints, please see my
>> > > > justification above. With Virtualization (look at KSM for example),
>> > > > shared pages are going to be increasingly important part of the
>> > > > accounting.
>> > > >
>> > >
>> > > Considering KSM, your cuounting style is tooo bad.
>> > >
>> > > You should add
>> > >
>> > >  - MEM_CGROUP_STAT_SHARED_BY_KSM
>> > >  - MEM_CGROUP_STAT_FOR_TMPFS/SYSV_IPC_SHMEM
>> > >
>>
>> No.. I am just talking about shared memory being important and shared
>> accounting being useful, no counters for KSM in particular (in the
>> memcg context).
>>
> Think so ? The number of memcg-private pages is in interest in my point of view.
>
> Anyway, I don't change my opinion as "sum of rss" is not necessary to be calculated
> in the kernel.
> If you want to provide that in memcg, please add it to global VM as /proc/meminfo.
>
> IIUC, KSM/SHMEM has some official method in global VM.
>

Kamezawa-San,

I implemented the same in user space and I get really bad results, here is why

1. I need to hold and walk the tasks list in cgroups and extract RSS
through /proc (results in worse hold times for the fork() scenario you
menioned)
2. The data is highly inconsistent due to the higher margin of error
in accumulating data which is changing as we run. By the time we total
and look at the memcg data, the data is stale

Would you be OK with the patch, if I renamed "shared_usage_in_bytes"
to "non_private_usage_in_bytes"?

Given that the stat is user initiated, I don't see your concern w.r.t.
overhead. Many subsystems like KSM do pay the overhead cost if the
user really wants the feature or the data. I would be really
interested in other opinions as well (if people do feel strongly
against or for the feature)

Balbir Singh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/