linux-kernel - Re: [RFC PATCH] mm/vmalloc: fix vmalloc which may return null if called with __GFP

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Wed, 8 May 2024 23:31:36 +0800
From: Hailong Liu <hailong.liu@...o.com>
To: Gao Xiang <hsiangkao@...ux.alibaba.com>
CC: <akpm@...ux-foundation.org>, Michal Hocko <mhocko@...e.com>,
	<urezki@...il.com>, <hch@...radead.org>, <lstoakes@...il.com>,
	<21cnbao@...il.com>, <linux-mm@...ck.org>, <linux-kernel@...r.kernel.org>,
	<xiang@...nel.org>, <chao@...nel.org>, Oven <liyangouwen1@...o.com>
Subject: Re: [RFC PATCH] mm/vmalloc: fix vmalloc which may return null if
 called with __GFP_NOFAIL

On Wed, 08. May 23:10, Gao Xiang wrote:
> Hi,
>
> On 2024/5/8 22:43, Hailong Liu wrote:
> > On Wed, 08. May 21:41, Gao Xiang wrote:
> > >
> > > +Cc Michal,
> > >
> > > On 2024/5/8 20:58, hailong.liu@...o.com wrote:
> > > > From: "Hailong.Liu" <hailong.liu@...o.com>
> > > >
> > > > Commit a421ef303008 ("mm: allow !GFP_KERNEL allocations for kvmalloc")
> > > > includes support for __GFP_NOFAIL, but it presents a conflict with
> > > > commit dd544141b9eb ("vmalloc: back off when the current task is
> > > > OOM-killed"). A possible scenario is as belows:
> > > >
> > > > process-a
> > > > kvcalloc(n, m, GFP_KERNEL | __GFP_NOFAIL)
> > > >       __vmalloc_node_range()
> > > > 	__vmalloc_area_node()
> > > > 	    vm_area_alloc_pages()
> > > >               --> oom-killer send SIGKILL to process-a
> > > >               if (fatal_signal_pending(current)) break;
> > > > --> return NULL;
> > > >
> > > > to fix this, do not check fatal_signal_pending() in vm_area_alloc_pages()
> > > > if __GFP_NOFAIL set.
> > > >
> > > > Reported-by: Oven <liyangouwen1@...o.com>
> > > > Signed-off-by: Hailong.Liu <hailong.liu@...o.com>
> > >
> > > Why taging this as RFC here?  It seems a corner-case fix of
> > > commit a421ef303008
> > >
> > > Thanks,
> > > Gao Xiang
> > >
> >
> > Hi Gao Xiang:
> >
> > RFC here to wait for a better way to handle this case :).
> > IMO, if vmalloc support __GFP_NOFAIL it should not return
> > null even system is deadlock on memory.
>
> The starting point is that kmalloc doesn't support __GFP_NOFAIL
> if order > 1 (even for very short temporary uses), see:
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/page_alloc.c?h=v6.8#n2896
>
> but it is possible if we have such page pointer array (since two
> (order-1) pages can only keep 1024 8-byte entries, it can happen
> if compression ratios are high), and kvmalloc(__GFP_NOFAIL) has
> already been supported for almost two years, it will fallback to
> order-0 allocation as described in commit e9c3cda4d86e
> ("mm, vmalloc: fix high order __GFP_NOFAIL allocations").
>
> With my limited understanding, I'm not sure why it can cause
> deadlock here since it will fallback to order-0 allocation then,
> and such allocation is just for short temporary uses again
> because kmalloc doesn't support order > 1 short memory
> allocation strictly.
>

deadlock on memory meands there is a memory leak causing
system to be unable to allocate memory not actual
*deadlock*.

> Thanks,
> Gao Xiang
>

--

Best Regards,
Hailong.