linux-kernel - Re: [PATCH/RFCv4 0/6] The Contiguous Memory Allocator framework

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AANLkTinPaq+0MbdW81uoc5_OZ=1Gy_mVYEBnwv8zgOBd@mail.gmail.com>
Date:	Thu, 26 Aug 2010 11:49:26 +0900
From:	Minchan Kim <minchan.kim@...il.com>
To:	Jonathan Corbet <corbet@....net>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Peter Zijlstra <peterz@...radead.org>,
	Michal Nazarewicz <m.nazarewicz@...sung.com>,
	linux-mm@...ck.org, Daniel Walker <dwalker@...eaurora.org>,
	FUJITA Tomonori <fujita.tomonori@....ntt.co.jp>,
	Hans Verkuil <hverkuil@...all.nl>,
	Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
	Kyungmin Park <kyungmin.park@...sung.com>,
	Marek Szyprowski <m.szyprowski@...sung.com>,
	Mark Brown <broonie@...nsource.wolfsonmicro.com>,
	Pawel Osciak <p.osciak@...sung.com>,
	Russell King <linux@....linux.org.uk>,
	Zach Pfeffer <zpfeffer@...eaurora.org>,
	linux-arm-kernel@...ts.infradead.org, linux-kernel@...r.kernel.org,
	linux-media@...r.kernel.org, Mel Gorman <mel@....ul.ie>
Subject: Re: [PATCH/RFCv4 0/6] The Contiguous Memory Allocator framework

On Thu, Aug 26, 2010 at 8:31 AM, Jonathan Corbet <corbet@....net> wrote:
> On Wed, 25 Aug 2010 15:58:14 -0700
> Andrew Morton <akpm@...ux-foundation.org> wrote:
>
>> > If you want guarantees you can free stuff, why not add constraints to
>> > the page allocation type and only allow MIGRATE_MOVABLE pages inside a
>> > certain region, those pages are easily freed/moved aside to satisfy
>> > large contiguous allocations.
>>
>> That would be good.  Although I expect that the allocation would need
>> to be 100% rock-solid reliable, otherwise the end user has a
>> non-functioning device.  Could generic core VM provide the required level
>> of service?
>
> The original OLPC has a camera controller which requires three contiguous,
> image-sized buffers in memory.  That system is a little memory constrained
> (OK, it's desperately short of memory), so, in the past, the chances of
> being able to allocate those buffers anytime some kid decides to start
> taking pictures was poor.  Thus, cafe_ccic.c has an option to snag the
> memory at initialization time and never let go even if you threaten its
> family.  Hell hath no fury like a little kid whose new toy^W educational
> tool stops taking pictures.
>
> That, of course, is not a hugely efficient use of memory on a
> memory-constrained system.  If the VM could reliably satisfy those
> allocation requestss, life would be wonderful.  Seems difficult.  But it
> would be a nicer solution than CMA, which, to a great extent, is really
> just a standardized mechanism for grabbing memory and never letting go.
>
>> It would help (a lot) if we could get more attention and buyin and
>> fedback from the potential clients of this code.  rmk's feedback is
>> valuable.  Have we heard from the linux-media people?  What other
>> subsystems might use it?  ieee1394 perhaps?  Please help identify
>> specific subsystems and I can perhaps help to wake people up.
>
> If this code had been present when I did the Cafe driver, I would have used
> it.  I think it could be made useful to a number of low-end camera drivers
> if the videobuf layer were made to talk to it in a way which Just Works.
>
> With a bit of tweaking, I think it could be made useful in other
> situations: the viafb driver, for example, really needs an allocator for
> framebuffer memory and it seems silly to create one from scratch.  Of
> course, there might be other possible solutions, like adding a "zones"
> concept to LMB^W memblock.
>
> The problem which is being addressed here is real.
>
> That said, the complexity of the solution still bugs me a bit, and the core
> idea is still to take big chunks of memory out of service for specific
> needs.  It would be far better if the VM could just provide big chunks on
> demand.  Perhaps compaction and the pressures of making transparent huge
> pages work will get us there, but I'm not sure we're there yet.
>
> jon

I agree. compaction and movable zone will be one of good solutions.

If some driver needs big contiguous chunk to work, it should make sure
to be allowable to have memory size for it before going. To make sure
it, we have to consider compaction of ZONE_MOVABLE zone. But one of
problems is anonymous page which can be has a role of pinned page in
non-swapsystem. Even most of embedded system has no swap.
But it's not hard to solve it.

We needs Mel's opinion, too.

-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/