linux-kernel - Re: [PATCH] Repeated fork() causes SLAB to grow without bound

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <546CC0CD.40906@suse.cz>
Date:	Wed, 19 Nov 2014 17:09:49 +0100
From:	Vlastimil Babka <vbabka@...e.cz>
To:	Konstantin Khlebnikov <koct9i@...il.com>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	Rik van Riel <riel@...hat.com>,
	Michel Lespinasse <walken@...gle.com>,
	Hugh Dickins <hughd@...gle.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	Tim Hartrick <tim@...ecast.com>, Michal Hocko <mhocko@...e.cz>
Subject: Re: [PATCH] Repeated fork() causes SLAB to grow without bound

On 11/19/2014 03:36 PM, Konstantin Khlebnikov wrote:
> On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka <vbabka@...e.cz> wrote:
>> On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote:
>>> On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov <koct9i@...il.com> wrote:
>>>> On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton
>>>> <akpm@...ux-foundation.org> wrote:
>>>>> On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel <riel@...hat.com> wrote:
>>>>>
>>>>>> > Because of the serial forking there does indeed end up being an
>>>>>> > infinite number of vmas.  The initial vma can never be deleted
>>>>>> > (even though the initial parent process has long since terminated)
>>>>>> > because the initial vma is referenced by the children.
>>>>>>
>>>>>> There is a finite number of VMAs, but an infite number of
>>>>>> anon_vmas.
>>>>>>
>>>>>> Subtle, yet deadly...
>>>>>
>>>>> Well, we clearly have the data structures screwed up.  I've forgotten
>>>>> enough about this code for me to be unable to work out what the fixed
>>>>> up data structures would look like :( But surely there is some proper
>>>>> solution here.  Help?
>>>>
>>>> Not sure if it's right but probably we could reuse on fork an old anon_vma
>>>> from the chain if it's already lost all vmas which points to it.
>>>> For endlessly forking exploit this should work mostly like proposed patch
>>>> which stops branching after some depth but without magic constant.
>>>
>>> Something like this. I leave proper comment for tomorrow.
>>
>> Hmm I'm not sure that will work as it is. If I understand it correctly, your
>> patch can detect if the parent's anon_vma has no own references at the fork()
>> time. But at the fork time, the parent is still alive, it only exits after the
>> fork, right? So I guess it still has own references and the child will still
>> allocate its new anon_vma, and the problem is not solved.
> 
> But it could reuse anon_vma from grandparent or older.
> Count of anon_vmas in chain will be limited with count of alive processes.

Ah I missed that it can reuse older anon_vma, sorry.

> I think it's better to describe this in terms of sets of anon_vma
> instead hierarchy:
> at clone vma inherits pages from parent together with set of anon_vma
> which they belong.
> For new pages it might allocate new anon_vma or reuse existing. After
> my patch vma
> will try to reuse anon_vma from that set which has no vmas which points to it.
> As a result there will be no parent-child relation between anon_vma and
> multiple pages might have equal (anon_vma, index) pair but I see no
> problems here.

Hmm I wonder if root anon_vma should be excluded from this reusal. For
performance reasons, exclusive pages go to non-root anon_vma (see
__page_set_anon_rmap()) and reusing root anon_vma would change this.
Also from reading http://lwn.net/Articles/383162/ I understand that correctness
also depends on the hierarchy and I wonder if there's a danger of reintroducing
a bug like the one described there.

Vlastimil

>>
>> So maybe we could detect that the own references dropped to zero when the parent
>> does exit, and then change mapping of all relevant pages to the root anon_vma,
>> destroy avc's of children and the anon_vma itself. But that sounds quite
>> heavyweight :/
>>
>> Vlastimil
>>
>>>>
>>>>>
>>
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@...ck.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@...ck.org"> email@...ck.org </a>
> 

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/