linux-kernel - Re: Regarding memory fragmentation using malloc....

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <858878.89812.qm@web162015.mail.bf1.yahoo.com>
Date:	Thu, 14 Apr 2011 05:24:56 -0700 (PDT)
From:	Pintu Agarwal <pintu_agarwal@...oo.com>
To:	Américo Wang <xiyou.wangcong@...il.com>,
	Michal Nazarewicz <mina86@...a86.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Eric Dumazet <eric.dumazet@...il.com>,
	Changli Gao <xiaosuo@...il.com>, Jiri Slaby <jslaby@...e.cz>,
	azurIt <azurit@...ox.sk>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, linux-fsdevel@...r.kernel.org,
	Jiri Slaby <jirislaby@...il.com>
Subject: Re: Regarding memory fragmentation using malloc....

Hello Mr. Michal,

Thanks for your comments.
Sorry. There was a small typo in my last sentence (mitigating not *migitating* memory fragmentation)
That means how can I measure the memory fragmentation either from user space or from kernel space.
Is there a way to measure the amount of memory fragmentation in linux?
Can you provide me some references for that?


Thanks,
Pintu



--- On Thu, 4/14/11, Michal Nazarewicz <mina86@...a86.com> wrote:

> From: Michal Nazarewicz <mina86@...a86.com>
> Subject: Re: Regarding memory fragmentation using malloc....
> To: "Américo Wang" <xiyou.wangcong@...il.com>, "Pintu Agarwal" <pintu_agarwal@...oo.com>
> Cc: "Andrew Morton" <akpm@...ux-foundation.org>, "Eric Dumazet" <eric.dumazet@...il.com>, "Changli Gao" <xiaosuo@...il.com>, "Jiri Slaby" <jslaby@...e.cz>, "azurIt" <azurit@...ox.sk>, linux-kernel@...r.kernel.org, linux-mm@...ck.org, linux-fsdevel@...r.kernel.org, "Jiri Slaby" <jirislaby@...il.com>
> Date: Thursday, April 14, 2011, 5:47 AM
> On Thu, 14 Apr 2011 08:44:50 +0200,
> Pintu Agarwal <pintu_agarwal@...oo.com>
> wrote:
> > As I can understand from your comments that, malloc
> from user space will not have much impact on memory
> fragmentation.
> 
> It has an impact, just like any kind of allocation, it just
> don't care about
> fragmentation of physical memory.  You can have only
> 0-order pages and
> successfully allocate megabytes of memory with malloc().
> 
> > Will the memory fragmentation be visible if I do
> kmalloc from
> > the kernel module????
> 
> It will be more visible in the sense that if you allocate 8
> KiB, kernel will
> have to find 8 KiB contiguous physical memory (ie. 1-order
> page).
> 
> >> No.  When you call malloc() only virtual
> address space is allocated.
> >> The actual allocation of physical space occurs
> when user space accesses
> >> the memory (either reads or writes) and it happens
> page at a time.
> > 
> > Here, if I do memset then I am accessing the
> memory...right? That I am doing already in my sample
> program.
> 
> Yes.  But note that even though it's a single memset()
> call, you are
> accessing page at a time and kernel is allocating page at a
> time.
> 
> On some architectures (not ARM) you could access two pages
> with a single
> instructions but I think that would result in two page
> faults anyway.  I
> might be wrong though, the details are not important
> though.
> 
> >> what really happens is that kernel allocates the
> 0-order
> >> pages and when
> >> it runs out of those, splits a 1-order page into
> two
> >> 0-order pages and
> >> takes one of those.
> > 
> > Actually, if I understand buddy allocator, it
> allocates pages from top to bottom.
> 
> No.  If you want to allocate a single 0-order page,
> buddy looks for a
> a free 0-order page.  If one is not found, it will
> look for 1-order page
> and split it.  This goes up till buddy reaches
> (MAX_ORDER-1)-page.
> 
> > Is the memory fragmentation is always a cause of the
> kernel space program and not user space at all?
> 
> Well, no.  If you allocate memory in user space,
> kernel will have to
> allocate physical memory and *every* allocation may
> contribute to
> fragmentation.  The point is, that all allocations
> from user-space are
> single-page allocations even if you malloc() MiBs of
> memory.
> 
> > Can you provide me with some references for migitating
> memory fragmentation in linux?
> 
> I'm not sure what you mean by that.
> 
> --Best regards,           
>                
>              _ 
>    _
> .o. | Liege of Serenely Enlightened Majesty of   
>   o' \,=./ `o
> ..o | Computer Science,  Michal "mina86"
> Nazarewicz    (o o)
> ooo +-----<email/xmpp: mnazarewicz@...gle.com>-----ooO--(_)--Ooo--
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm'
> in
> the body to majordomo@...ck.org. 
> For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@...ck.org">
> email@...ck.org
> </a>
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/