linux-kernel - Re: [PATCH 3/4] writeback: nr_dirtied and nr_entered

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20100824032534.GC11970@localhost>
Date:	Tue, 24 Aug 2010 11:25:34 +0800
From:	Wu Fengguang <fengguang.wu@...el.com>
To:	Michael Rubin <mrubin@...gle.com>
Cc:	Shailabh Nagar <nagar@...son.ibm.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"jack@...e.cz" <jack@...e.cz>, "riel@...hat.com" <riel@...hat.com>,
	"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
	"david@...morbit.com" <david@...morbit.com>,
	"npiggin@...nel.dk" <npiggin@...nel.dk>, "hch@....de" <hch@....de>,
	"axboe@...nel.dk" <axboe@...nel.dk>
Subject: Re: [PATCH 3/4] writeback: nr_dirtied and nr_entered_writeback in
 /proc/vmstat

On Tue, Aug 24, 2010 at 11:02:42AM +0800, Michael Rubin wrote:
> On Mon, Aug 23, 2010 at 7:30 PM, Wu Fengguang <fengguang.wu@...el.com> wrote:
> > It's about the interface, I don't mind you adding the per-node vmstat
> > entries which may be convenient for you and mostly harmless to others.
> >
> > My concern is, what do you think about the existing
> > /proc/<pid>/io:write_bytes interface and is it good enough for your?
> > You'll have to iterate through tasks to collect numbers for one job or
> > for the whole system, however that should be easy and more flexible?
> 
> Is this as an alternative to the vmstat counters or the per node
> vmstat counters?
> 
> In either case I am not sure /proc/pid will be sufficient. What if 20
> processes are created, write a lot of data, then quit? How do we know
> about those events later? What about jobs that wake up, write data
> then quit quickly?

According to Documentation/accounting/taskstats.txt, it's possible to
register a task for collecting its stat numbers when it quits. However
I'm not sure if there are reliable ways to do this if tasks come and
go very quickly. It may help to somehow automatically add the numbers
to the parent process at process quit time.

> If we are running many many tasks I would think the amount of time it
> can take to cycle through all of them might mean we might not capture
> all the data.

Yes, 10k tasks may be too much to track.

> The goal is to have a complete view of the dirtying and cleaning of
> the pages over time. Using /proc/pid feels like it won't achieve that.

Looks so.

Thanks,
Fengguang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/