linux-kernel - Re: KSM: current madvise rollup

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 11 Jul 2009 01:42:25 +0300
From:	Izik Eidus <ieidus@...hat.com>
To:	Hugh Dickins <hugh.dickins@...cali.co.uk>
CC:	Andrea Arcangeli <aarcange@...hat.com>,
	Rik van Riel <riel@...hat.com>,
	Chris Wright <chrisw@...hat.com>,
	Nick Piggin <nickpiggin@...oo.com.au>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: KSM: current madvise rollup

Hugh Dickins wrote:
>  
>   
Hey Hugh,

I started to hack the code around to make sure i understand everything, 
and in addition i wanted to add few things

One thing that catched my eyes was:
> +
> +/*
> + * cmp_and_merge_page - take a page computes its hash value and check if there
> + * is similar hash value to different page,
> + * in case we find that there is similar hash to different page we call to
> + * try_to_merge_two_pages().
> + *
> + * @page: the page that we are searching identical page to.
> + * @rmap_item: the reverse mapping into the virtual address of this page
> + */
> +static void cmp_and_merge_page(struct page *page, struct rmap_item *rmap_item)
> +{
> +	struct page *page2[1];
> +	struct rmap_item *tree_rmap_item;
> +	unsigned int checksum;
> +	int err;
> +
> +	if (in_stable_tree(rmap_item))
> +		remove_rmap_item_from_tree(rmap_item);
> +
>
>   


So when we enter to cmp_and_merge_page, if the page is in_stable_tree() 
we will remove the rmap_item from the stable tree,
And then we have inside:

> + * ksm_do_scan  - the ksm scanner main worker function.
> + * @scan_npages - number of pages we want to scan before we return.
> + */
> +static void ksm_do_scan(unsigned int scan_npages)
> +{
> +	struct rmap_item *rmap_item;
> +	struct page *page;
> +
> +	while (scan_npages--) {
> +		cond_resched();
> +		rmap_item = scan_get_next_rmap_item(&page);
> +		if (!rmap_item)
> +			return;
> +		if (!PageKsm(page) || !in_stable_tree(rmap_item))
> +			cmp_and_merge_page(page, rmap_item);
> +		put_page(page);
> +	}
> +}
>   

So this check for: if (!PageKsm(page) || !in_stable_tree(rmap_item)) 
will be true for !in_stable_tree(rmap_item)

Isnt it mean that we are "stop using the stable tree help" ?
It look like every item that will go into the stable tree will get 
flushed from it in the second run, that will highly increase the ksmd 
cpu usage, and will make it find less pages...
Was this what you wanted to do? or am i missed anything?

Beside this one more thing i noticed while checking this code:
beacuse the new "Ksm shared page" is not File backed page, it isnt count 
in top as a shared page, and i couldnt find a way to see how many pages 
are shared for each application..
This is important for management tools such as a tool that will want to 
know what Virtual Machines it want to migrate from the host into another 
host based on the memory sharing in that specific host (Meaning how much 
ram it really take on that specific host)

So I started to prepre a patch that will show merged pages count inside 
/proc/pid/mergedpages, But then i thought this statics lie:
if we will have 2 applications: application A and application B, that 
share the same page, how should it look like?:

cat /proc/pid_of_A/merged_pages -> 1
cat /proc/pid_of_B/merged_pages -> 1

or:

cat /proc/pid_of_A/merged_pages -> 0 (beacuse this one was shared with 
the page of B)
cat /proc/pid_of_B/merged_pages -> 1

To make the second method thing work as much as reaible as we can we 
would want to break KsmPages that have just one mapping into them...


What do you think about that? witch direction should we take for that?


(Other than this stuff, everything running happy and nice, I think cpu 
is little bit too high beacuse the removing of the stable_tree issue)

Thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/