linux-kernel - Re: [RFC PATCH -tip 0/2 v3] pagecache tracepoints proposal

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20100208130412.GB24467@balbir.in.ibm.com>
Date:	Mon, 8 Feb 2010 18:34:12 +0530
From:	Balbir Singh <balbir@...ux.vnet.ibm.com>
To:	Keiichi KII <k-keiichi@...jp.nec.com>
Cc:	linux-kernel@...r.kernel.org, mingo@...e.hu, lwoodman@...hat.com,
	linux-mm@...ck.org, Tom Zanussi <tzanussi@...il.com>,
	riel@...hat.com, rostedt@...dmis.org, akpm@...ux-foundation.org,
	fweisbec@...il.com, Munehiro Ikeda <m-ikeda@...jp.nec.com>,
	Atsushi Tsuji <a-tsuji@...jp.nec.com>
Subject: Re: [RFC PATCH -tip 0/2 v3] pagecache tracepoints proposal

* Keiichi KII <k-keiichi@...jp.nec.com> [2010-02-04 21:17:35]:

> Hello,
> 
> This is v3 of a patchset to add some tracepoints for pagecache.
> 
> I would propose several tracepoints for tracing pagecache behavior and
> a script for these.
> By using both the tracepoints and the script, we can analysis pagecache behavior
> like usage or hit ratio with high resolution like per process or per file. 
> Example output of the script looks like:
> 
> [process list]
> o yum-3215
>                           cache find  cache hit  cache hit
>         device      inode      count      count      ratio
>   --------------------------------------------------------
>          253:0         16      34434      34130     99.12%
>          253:0        198       9692       9463     97.64%
>          253:0        639        647        628     97.06%
>          253:0        778         32         29     90.62%
>          253:0       7305      50225      49005     97.57%
>          253:0     144217         12         10     83.33%
>          253:0     262775         16         13     81.25%
> *snip*

Very nice, we should be able to sum these to get a system wide view

> 
> -------------------------------------------------------------------------------
> 
> [file list]
>         device              cached
>      (maj:min)      inode    pages
>   --------------------------------
>          253:0         16     5752
>          253:0        198     2233
>          253:0        639       51
>          253:0        778       86
>          253:0       7305    12307
>          253:0     144217       11
>          253:0     262775       39
> *snip*
> 
> [process list]
> o yum-3215
>         device              cached    added  removed      indirect
>      (maj:min)      inode    pages    pages    pages removed pages
>   ----------------------------------------------------------------
>          253:0         16    34130     5752        0             0
>          253:0        198     9463     2233        0             0
>          253:0        639      628       51        0             0
>          253:0        778       29       78        0             0
>          253:0       7305    49005    12307        0             0
>          253:0     144217       10       11        0             0
>          253:0     262775       13       39        0             0
> *snip*
>   ----------------------------------------------------------------
>   total:                    102346    26165        1             0
                                                    ^^^
                                                Is this 1 stray?
> 
> We can now know system-wide pagecache usage by /proc/meminfo.
> But we have no method to get higher resolution information like per file or
> per process usage than system-wide one.

It would be really nice to see if we can detect the mapped from the
unmapped page cache

> A process may share some pagecache or add a pagecache to the memory or
> remove a pagecache from the memory.
> If a pagecache miss hit ratio rises, maybe it leads to extra I/O and
> affects system performance.
> 
> So, by using the tracepoints we can get the following information.
>  1. how many pagecaches each process has per each file
>  2. how many pages are cached per each file
>  3. how many pagecaches each process shares
>  4. how often each process adds/removes pagecache
>  5. how long a pagecache stays in the memory
>  6. pagecache hit rate per file
> 
> Especially, the monitoring pagecache usage per each file and pagecache hit 
> ratio would help us tune some applications like database.
> And it will also help us tune the kernel parameters like "vm.dirty_*".
> 
> Changelog since v2
>   o add new script to monitor pagecache hit ratio per process.
>   o use DECLARE_EVENT_CLASS
> 
> Changelog since v1
>   o Add a script based on "perf trace stream scripting support".
> 
> Any comments are welcome.

-- 
	Three Cheers,
	Balbir
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/