linux-kernel - Re: [RFC] can we use vmalloc to alloc thread stack if compaction failed

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 28 Jul 2016 16:45:06 +0800
From:	Xishi Qiu <qiuxishi@...wei.com>
To:	Michal Hocko <mhocko@...nel.org>
CC:	Tejun Heo <tj@...nel.org>, Ingo Molnar <mingo@...nel.org>,
	Peter Zijlstra <peterz@...radead.org>,
	LKML <linux-kernel@...r.kernel.org>,
	Linux MM <linux-mm@...ck.org>,
	Andy Lutomirski <luto@...capital.net>,
	Yisheng Xie <xieyisheng1@...wei.com>
Subject: Re: [RFC] can we use vmalloc to alloc thread stack if compaction
 failed

On 2016/7/28 15:58, Michal Hocko wrote:

> On Thu 28-07-16 15:41:53, Xishi Qiu wrote:
>> On 2016/7/28 15:20, Michal Hocko wrote:
>>
>>> On Thu 28-07-16 15:08:26, Xishi Qiu wrote:
>>>> Usually THREAD_SIZE_ORDER is 2, it means we need to alloc 16kb continuous
>>>> physical memory during fork a new process.
>>>>
>>>> If the system's memory is very small, especially the smart phone, maybe there
>>>> is only 1G memory. So the free memory is very small and compaction is not
>>>> always success in slowpath(__alloc_pages_slowpath), then alloc thread stack
>>>> may be failed for memory fragment.
>>>
>>> Well, with the current implementation of the page allocator those
>>> requests will not fail in most cases. The oom killer would be invoked in
>>> order to free up some memory.
>>>
>>
>> Hi Michal,
>>
>> Yes, it success in most cases, but I did have seen this problem in some
>> stress-test.
>>
>> DMA free:470628kB, but alloc 2 order block failed during fork a new process.
>> There are so many memory fragments and the large block may be soon taken by
>> others after compact because of stress-test.
>>
>> --- dmesg messages ---
>> 07-13 08:41:51.341 <4>[309805.658142s][pid:1361,cpu5,sManagerService]sManagerService: page allocation failure: order:2, mode:0x2000d1
> 
> Yes but this is __GFP_DMA allocation. I guess you have already reported
> this failure and you've been told that this is quite unexpected for the
> kernel stack allocation. It is your out-of-tree patch which just makes
> things worse because DMA restricted allocations are considered "lowmem"
> and so they do not invoke OOM killer and do not retry like regular
> GFP_KERNEL allocations.

Hi Michal,

Yes, we add GFP_DMA, but I don't think this is the key for the problem.

If we do oom-killer, maybe we will get a large block later, but there
is enough free memory before oom(although most of them are fragments).

I wonder if we can alloc success without kill any process in this situation.
Maybe use vmalloc is a good way, but I don't know the influence.

Thanks,
Xishi Qiu