linux-kernel - Re: [RFC] propagate gfp_t to page table alloc functions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 24 Apr 2012 17:01:16 +0900
From:	Minchan Kim <minchan@...nel.org>
To:	Nick Piggin <npiggin@...il.com>
CC:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Ingo Molnar <mingo@...hat.com>, x86@...nel.org,
	Hugh Dickins <hughd@...gle.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Rik van Riel <riel@...hat.com>, Mel Gorman <mgorman@...e.de>,
	kosaki.motohiro@...fujitsu.com, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org
Subject: Re: [RFC] propagate gfp_t to page table alloc functions

On 04/24/2012 04:48 PM, Nick Piggin wrote:

> On 24 April 2012 17:19, Minchan Kim <minchan@...nel.org> wrote:
>> On 04/24/2012 03:13 PM, Nick Piggin wrote:
>>
>>> 2012/4/24 Minchan Kim <minchan@...nel.org>:
>>>> On 04/24/2012 02:16 PM, KAMEZAWA Hiroyuki wrote:
>>>>
>>>>> (2012/04/23 17:55), Minchan Kim wrote:
>>>>>
>>>>>> As I test some code, I found a problem about deadlock by lockdep.
>>>>>> The reason I saw the message is __vmalloc calls map_vm_area which calls
>>>>>> pud/pmd_alloc without gfp_t. so although we call __vmalloc with
>>>>>> GFP_ATOMIC or GFP_NOIO, it ends up allocating pages with GFP_KERNEL.
>>>>>> The should be a BUG. This patch fixes it by passing gfp_to to low page
>>>>>> table allocate functions.
>>>>>>
>>>>>> Signed-off-by: Minchan Kim <minchan@...nel.org>
>>>>>
>>>>>
>>>>> Hmm ? vmalloc should support GFP_ATOMIC ?
>>>>
>>>>
>>>> I'm not sure but alloc_large_system_hash already has used.
>>>> And it's not specific on GFP_ATOMIC.
>>>> We have to care of GFP_NOFS and GFP_NOIO to prevent deadlock on reclaim
>>>> context.
>>>> There are some places to use GFP_NOFS and we don't emit any warning
>>>> message in case of that.
>>>
>>> What's the lockdep warning?
>>
>>
>> It's just some private-test code, not-mainlined and lockdep warning is like this.
>>
>> [ INFO: inconsistent lock state ]
>> 3.4.0-rc3-next-20120417+ #80 Not tainted
>> ---------------------------------
>> inconsistent {RECLAIM_FS-ON-W} -> {IN-RECLAIM_FS-R} usage.
>>
>> It seems test code calls vmalloc inside reclaim context so that it enters
>> reclaim context, again by map_vm_area which allocates pages with GFP_KERNEL.
>>
>> Of course, I can avoid this problem by fixing the caller but during I look into
>> this problem, found other places to use gfp_t with "context restriction".
>>
>>
>>>
>>
>>> vmalloc was never supposed to use gfp flags for allocation "context"
>>> restriction. I.e., it
>>> was always supposed to have blocking, fs, and io capable allocation
>>> context. The flags
>>> were supposed to be a memory type modifier.
>>
>>
>> You mean "zone modifiers"?
> 
> Yeah, things like that.
> 
>>> These different classes of flags is a bit of a problem and source of
>>> confusion we have.
>>> We should be doing more checks for them, of course.
>>
>>
>> It might need some warning in __vmalloc and family which use gfp_t
>> if the caller use context flags.
> 
> I think that would be a good idea.
> 
> 
>>> I suspect you need to fix the caller?
>>
>>
>> Hmm, there are several places to use GFP_NOIO and GFP_NOFS even, GFP_ATOMIC.
>> I believe it's not trivial now.
> 
> They're all buggy then. Unfortunately not through any real fault of their own.


That's why I send it with RFC before I have to make all architecture change.
Nick, Thanks!

> 
> I would say add a bit of warnings and documentation, and see what can be done
> about callers.

> 

> We should not take lightly the decision to make the API more permissive, because
> as you can see it's more work for implementation. Making it ATOMIC safe is even


Agree. Will add waring and Cced all maintainers.

> harder, requiring irqsafe locks and such, and it might be tricky for some


irqsafe? Why should we consider it? 
Just out of curiosity.

-- 
Kind regards,
Minchan Kim
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/