linux-kernel - Re: [PATCH 4/4] maps: /proc/<pid>/pmaps interface

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070817165808.GM30556@waste.org>
Date:	Fri, 17 Aug 2007 11:58:08 -0500
From:	Matt Mackall <mpm@...enic.com>
To:	Fengguang Wu <wfg@...l.ustc.edu.cn>
Cc:	Andrew Morton <akpm@...l.org>,
	Jeremy Fitzhardinge <jeremy@...p.org>,
	David Rientjes <rientjes@...gle.com>,
	John Berthels <jjberthels@...il.com>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	linux-kernel@...r.kernel.org
Subject: Re: [PATCH 4/4] maps: /proc/<pid>/pmaps interface - memory maps in granularity of pages

On Fri, Aug 17, 2007 at 02:47:27PM +0800, Fengguang Wu wrote:
> Matt,
> 
> It's not easy to do direct performance comparisons between pmaps and
> pagemap/kpagemap. However some close analyzes are still possible :)
> 
> 1) code size
> pmaps                   ~200 LOC
> pagemap/kpagemap        ~300 LOC
> 
> 2) dataset size
> take for example my running firefox on Intel Core 2:
> VSZ             400 MB
> RSS              64 MB, or 16k pages
> pmaps            64 KB, wc shows 2k lines, or so much page ranges
> pagemap         800 KB, could be heavily optimized by returning partial data

I take it you're in 64-bit mode?

You're right, this data compresses well in many circumstances. I
suspect it will suffer under memory pressure though. That will
fragment the ranges in-memory and also fragment the active bits. The
worst case here is huge, of course, but realistically I'd expect
something like 2x-4x.

But there are still the downsides I have mentioned:

- you don't get page frame numbers
- you can't do random access

And how long does it take to pull the data out? My benchmarks show
greater than 50MB/s (and that's with the version in -mm that's doing
double buffering), so that 800K would take < .016s. 

> kpagemap        256 KB
> 
> 3) runtime overheads
> pmaps            2k lines of string processing(encode/decode)
> kpagemap        16k seek()/read()s, and context switches (could be
>                     optimized somehow by doing a PFN sort first, but
>                     that's also non-trivial overheads)

You can do anywhere between 16k small reads or 1 large read. Depends
what data you're trying to get. Right now, kpagemap is fast enough
that I can do realtime displays of the whole of memory in my desktop
in a GUI written in Python. And Python is fairly horrible for drawing
bitmaps and such.

http://www.selenic.com/Screenshot-kpagemap.png

> So pmaps seems to be a clear winner :)

Except that it's only providing a subset of the data.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/