linux-kernel - Re: Regarding memory fragmentation using malloc....

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <op.vtxb92f73l0zgt@mnazarewicz-glaptop>
Date:	Thu, 14 Apr 2011 12:47:53 +0200
From:	"Michal Nazarewicz" <mina86@...a86.com>
To:	Américo Wang <xiyou.wangcong@...il.com>,
	"Pintu Agarwal" <pintu_agarwal@...oo.com>
Cc:	"Andrew Morton" <akpm@...ux-foundation.org>,
	"Eric Dumazet" <eric.dumazet@...il.com>,
	"Changli Gao" <xiaosuo@...il.com>, "Jiri Slaby" <jslaby@...e.cz>,
	azurIt <azurit@...ox.sk>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, linux-fsdevel@...r.kernel.org,
	"Jiri Slaby" <jirislaby@...il.com>
Subject: Re: Regarding memory fragmentation using malloc....

On Thu, 14 Apr 2011 08:44:50 +0200, Pintu Agarwal  
<pintu_agarwal@...oo.com> wrote:
> As I can understand from your comments that, malloc from user space will  
> not have much impact on memory fragmentation.

It has an impact, just like any kind of allocation, it just don't care  
about
fragmentation of physical memory.  You can have only 0-order pages and
successfully allocate megabytes of memory with malloc().

> Will the memory fragmentation be visible if I do kmalloc from
> the kernel module????

It will be more visible in the sense that if you allocate 8 KiB, kernel  
will
have to find 8 KiB contiguous physical memory (ie. 1-order page).

>> No.  When you call malloc() only virtual address space is allocated.
>> The actual allocation of physical space occurs when user space accesses
>> the memory (either reads or writes) and it happens page at a time.
>
> Here, if I do memset then I am accessing the memory...right? That I am  
> doing already in my sample program.

Yes.  But note that even though it's a single memset() call, you are
accessing page at a time and kernel is allocating page at a time.

On some architectures (not ARM) you could access two pages with a single
instructions but I think that would result in two page faults anyway.  I
might be wrong though, the details are not important though.

>> what really happens is that kernel allocates the 0-order
>> pages and when
>> it runs out of those, splits a 1-order page into two
>> 0-order pages and
>> takes one of those.
>
> Actually, if I understand buddy allocator, it allocates pages from top  
> to bottom.

No.  If you want to allocate a single 0-order page, buddy looks for a
a free 0-order page.  If one is not found, it will look for 1-order page
and split it.  This goes up till buddy reaches (MAX_ORDER-1)-page.

> Is the memory fragmentation is always a cause of the kernel space  
> program and not user space at all?

Well, no.  If you allocate memory in user space, kernel will have to
allocate physical memory and *every* allocation may contribute to
fragmentation.  The point is, that all allocations from user-space are
single-page allocations even if you malloc() MiBs of memory.

> Can you provide me with some references for migitating memory  
> fragmentation in linux?

I'm not sure what you mean by that.

-- 
Best regards,                                         _     _
.o. | Liege of Serenely Enlightened Majesty of      o' \,=./ `o
..o | Computer Science,  Michal "mina86" Nazarewicz    (o o)
ooo +-----<email/xmpp: mnazarewicz@...gle.com>-----ooO--(_)--Ooo--
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/