| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <112566.51053.qm@web162019.mail.bf1.yahoo.com>
Date: Wed, 13 Apr 2011 06:56:00 -0700 (PDT)
From: Pintu Agarwal <pintu_agarwal@...oo.com>
To: Américo Wang <xiyou.wangcong@...il.com>
Cc: Andrew Morton <akpm@...ux-foundation.org>,
Eric Dumazet <eric.dumazet@...il.com>,
Changli Gao <xiaosuo@...il.com>, Jiri Slaby <jslaby@...e.cz>,
azurIt <azurit@...ox.sk>, linux-kernel@...r.kernel.org,
linux-mm@...ck.org, linux-fsdevel@...r.kernel.org,
Jiri Slaby <jirislaby@...il.com>
Subject: Re: Regarding memory fragmentation using malloc....
Hi,
My requirement is, I wanted to measure memory fragmentation level in linux kernel2.6.29 (ARM cortex A8 without swap).
How can I measure fragmentation level(percentage) from /proc/buddyinfo ?
Example : After each page allocation operation, I need to measure fragmentation level. If the level is above 80%, I will trigger a OOM or something to the user.
How can I reproduce this memory fragmentation scenario using a sample program?
Here is my sample program: (to check page allocation using malloc)
----------------------------------------------
#include<stdio.h>
#include<stdlib.h>
#include<string.h>
#include<errno.h>
#include<unistd.h>
#define PAGE_SIZE (4*1024)
#define MEM_BLOCK (64*PAGE_SIZE)
#define MAX_LIMIT (16)
int main()
{
char *ptr[MAX_LIMIT+1] = {NULL,};
int i = 0;
printf("Requesting <%d> blocks of memory of block size <%d>........\n",MAX_LIMIT,MEM_BLOCK);
system("cat /proc/buddyinfo");
system("cat /proc/zoneinfo | grep free_pages");
printf("*****************************************\n\n\n");
for(i=0; i<MAX_LIMIT; i++)
{
ptr[i] = (char *)malloc(sizeof(char)*MEM_BLOCK);
if(ptr[i] == NULL)
{
printf("ERROR : malloc failed(counter %d) <%s>\n",i,strerror(errno));
system("cat /proc/buddyinfo");
system("cat /proc/zoneinfo | grep free_pages");
printf("press any key to terminate......");
getchar();
exit(0);
}
memset(ptr[i],1,MEM_BLOCK);
sleep(1);
//system("cat /proc/buddyinfo");
//system("cat /proc/zoneinfo | grep free_pages");
//printf("-----------------------------------------\n");
}
sleep(1);
system("cat /proc/buddyinfo");
system("cat /proc/zoneinfo | grep free_pages");
printf("-----------------------------------------\n");
printf("press any key to end......");
getchar();
for(i=0; i<MAX_LIMIT; i++)
{
if(ptr[i] != NULL)
{
free(ptr[i]);
}
}
printf("DONE !!!\n");
return 0;
}
EACH BLOCK SIZE = 64 Pages ==> (64 * 4 * 1024)
TOTAL BLOCKS = 16
----------------------------------------------
In my linux2.6.29 ARM machine, the initial /proc/buddyinfo shows the following:
Node 0, zone DMA 17 22 1 1 0 1 1 0 0 0 0 0
Node 1, zone DMA 15 320 423 225 97 26 1 0 0 0 0 0
After running my sample program (with 16 iterations) the buddyinfo output is as follows:
Requesting <16> blocks of memory of block size <262144>........
Node 0, zone DMA 17 22 1 1 0 1 1 0 0 0 0 0
Node 1, zone DMA 15 301 419 224 96 27 1 0 0 0 0 0
nr_free_pages 169
nr_free_pages 6545
*****************************************
Node 0, zone DMA 17 22 1 1 0 1 1 0 0 0 0 0
Node 1, zone DMA 18 2 305 226 96 27 1 0 0 0 0 0
nr_free_pages 169
nr_free_pages 5514
-----------------------------------------
The requested block size is 64 pages (2^6) for each block.
But if we see the output after 16 iterations the buddyinfo allocates pages only from Node 1 , (2^0, 2^1, 2^2, 2^3).
But the actual allocation should happen from (2^6) block in buddyinfo.
Questions:
1) How to analyse buddyinfo based on each page block size?
2) How and in what scenario the buddyinfo changes?
3) Can we rely completely on buddyinfo information for measuring the level of fragmentation?
Can somebody through some more lights on this???
Thanks,
Pintu
--- On Wed, 4/13/11, Américo Wang <xiyou.wangcong@...il.com> wrote:
> From: Américo Wang <xiyou.wangcong@...il.com>
> Subject: Re: Regarding memory fragmentation using malloc....
> To: "Pintu Agarwal" <pintu_agarwal@...oo.com>
> Cc: "Andrew Morton" <akpm@...ux-foundation.org>, "Eric Dumazet" <eric.dumazet@...il.com>, "Changli Gao" <xiaosuo@...il.com>, "Jiri Slaby" <jslaby@...e.cz>, "azurIt" <azurit@...ox.sk>, linux-kernel@...r.kernel.org, linux-mm@...ck.org, linux-fsdevel@...r.kernel.org, "Jiri Slaby" <jirislaby@...il.com>
> Date: Wednesday, April 13, 2011, 6:44 AM
> On Wed, Apr 13, 2011 at 2:54 PM,
> Pintu Agarwal <pintu_agarwal@...oo.com>
> wrote:
> > Dear All,
> >
> > I am trying to understand how memory fragmentation
> occurs in linux using many malloc calls.
> > I am trying to reproduce the page fragmentation
> problem in linux 2.6.29.x on a linux mobile(without Swap)
> using a small malloc(in loop) test program of BLOCK_SIZE
> (64*(4*K)).
> > And then monitoring the page changes in
> /proc/buddyinfo after each operation.
> > From the output I can see that the page values under
> buddyinfo keeps changing. But I am not able to relate these
> changes with my malloc BLOCK_SIZE.
> > I mean with my BLOCK_SIZE of (2^6 x 4K ==> 2^6
> PAGES) the 2^6 th block under /proc/buddyinfo should change.
> But this is not the actual behaviour.
> > Whatever is the blocksize, the buddyinfo changes only
> for 2^0 or 2^1 or 2^2 or 2^3.
> >
> > I am trying to measure the level of fragmentation
> after each page allocation.
> > Can somebody explain me in detail, how actually
> /proc/buddyinfo changes after each allocation and
> deallocation.
> >
>
> What malloc() sees is virtual memory of the process, while
> buddyinfo
> shows physical memory pages.
>
> When you malloc() 64K memory, the kernel may not allocate a
> 64K
> physical memory at one time
> for you.
>
> Thanks.
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists