linux-kernel - Re: [RFC PATCH -tip 2/2 v2] add a scripts for pagecache usage per process

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4B5E1855.4090809@bx.jp.nec.com>
Date:	Mon, 25 Jan 2010 17:16:53 -0500
From:	Keiichi KII <k-keiichi@...jp.nec.com>
To:	Tom Zanussi <tzanussi@...il.com>
CC:	linux-kernel@...r.kernel.org, lwoodman@...hat.com,
	linux-mm@...ck.org, mingo@...e.hu, riel@...hat.com,
	rostedt@...dmis.org, akpm@...ux-foundation.org, fweisbec@...il.com,
	Munehiro Ikeda <m-ikeda@...jp.nec.com>,
	Atsushi Tsuji <a-tsuji@...jp.nec.com>
Subject: Re: [RFC PATCH -tip 2/2 v2] add a scripts for pagecache usage per
 process

(2010年01月23日 03:21), Tom Zanussi wrote:
> Hi,
> 
> On Fri, 2010-01-22 at 19:08 -0500, Keiichi KII wrote:
>> The scripts are implemented based on the trace stream scripting support.
>> And the scripts implement the following.
>>  - how many pagecaches each process has per each file
>>  - how many pages are cached per each file
>>  - how many pagecaches each process shares
>>
> 
> Nice, it looks like a very useful script - I gave it a quick try and it
> seems to work well...
> 
> The only problem I see, nothing to do with your script and nothing you
> can do anything about at the moment, is that the record step generates a
> huge amount of data, which of course makes the event processing take
> awhile.  A lot of it appears to be due to perf itself - being able to
> filter out the perf-generated events in the kernel would make a big
> difference, I think; you normally don't want to see those anyway...

Yes, right. I don't want to process the data of perf itself.
I will try to find any way to solve this problem. 

> BTW, I see that you did your first version in Python - not that you'd
> want to redo it again, but just FYI I now have working Python support
> that I'll be posting soon - I still have some small details to hammer
> out, but if you have any other scripts in the pipeline, in a couple days
> you'll be able to use Python instead if you want.

It will help me create some scripts. I will use Python support if it is posted.

>> To monitor pagecache usage per a process, run "pagecache-usage-record" to
>> record perf data for "pagecache-usage.pl" and run "pagecache-usage-report"
>> to display.
> 
> Another way of course would be to use 'perf trace record/report' and the
> script name as shown by perf trace -l:
> 
> $ perf trace record pagecache-usage
> $ perf trace report pagecache-usage

Thank you for your information.
I will use this way.

>> Index: linux-2.6-tip/tools/perf/scripts/perl/pagecache-usage.pl
>> ===================================================================
>> --- /dev/null
>> +++ linux-2.6-tip/tools/perf/scripts/perl/pagecache-usage.pl
>> @@ -0,0 +1,160 @@
>> +# perf trace event handlers, generated by perf trace -g perl
> 
> You might want to get rid of this and add a short description and your
> name, if you want to take credit for it. ;-)
> 
>> +# Licensed under the terms of the GNU GPL License version 2
>> +
> 
>> +# The common_* event handler fields are the most useful fields common to
>> +# all events.  They don't necessarily correspond to the 'common_*' fields
>> +# in the format files.  Those fields not available as handler params can
>> +# be retrieved using Perl functions of the form common_*($context).
>> +# See Context.pm for the list of available functions.
>> +
> 
> You can get rid of this part too - it's just meant to be helpful
> information generated when starting a script.
> 

I'll remove unnecessary comments and add a short description next time.

>> +my %unhandled;
>> +
>> +sub trace_unhandled
>> +{
>> +	my ($event_name, $context, $common_cpu, $common_secs, $common_nsecs,
>> +	    $common_pid, $common_comm) = @_;
>> +
>> +	$unhandled{$event_name}++;
>> +}
>> +
>> +sub print_unhandled
>> +{
>> +	if ((scalar keys %unhandled) == 0) {
>> +	    print "unhandled events nothing\n";
> 
> This is kind of distracting - it's not too useful to know that you don't
> have unhandled events, but if you do have some, it is useful to print
> those as you do below - it points out that some event type are being
> unnecessarily recorded or the script is being run on the wrong trace
> data.

I don't have unhandled events to monitor pagecache usage as you say.
So I will remove these code.

Thanks,
Keiichi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/