linux-kernel - Re: [PATCH] Repeated fork() causes SLAB to grow without bound

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 19 Nov 2014 18:36:45 +0400
From:	Konstantin Khlebnikov <koct9i@...il.com>
To:	Vlastimil Babka <vbabka@...e.cz>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Rik van Riel <riel@...hat.com>,
	Michel Lespinasse <walken@...gle.com>,
	Hugh Dickins <hughd@...gle.com>,
	Andrea Arcangeli <aarcange@...hat.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	Tim Hartrick <tim@...ecast.com>, Michal Hocko <mhocko@...e.cz>
Subject: Re: [PATCH] Repeated fork() causes SLAB to grow without bound

On Wed, Nov 19, 2014 at 2:50 AM, Vlastimil Babka <vbabka@...e.cz> wrote:
> On 11/19/2014 12:02 AM, Konstantin Khlebnikov wrote:
>> On Wed, Nov 19, 2014 at 1:15 AM, Konstantin Khlebnikov <koct9i@...il.com> wrote:
>>> On Tue, Nov 18, 2014 at 11:19 PM, Andrew Morton
>>> <akpm@...ux-foundation.org> wrote:
>>>> On Mon, 17 Nov 2014 21:41:57 -0500 Rik van Riel <riel@...hat.com> wrote:
>>>>
>>>>> > Because of the serial forking there does indeed end up being an
>>>>> > infinite number of vmas.  The initial vma can never be deleted
>>>>> > (even though the initial parent process has long since terminated)
>>>>> > because the initial vma is referenced by the children.
>>>>>
>>>>> There is a finite number of VMAs, but an infite number of
>>>>> anon_vmas.
>>>>>
>>>>> Subtle, yet deadly...
>>>>
>>>> Well, we clearly have the data structures screwed up.  I've forgotten
>>>> enough about this code for me to be unable to work out what the fixed
>>>> up data structures would look like :( But surely there is some proper
>>>> solution here.  Help?
>>>
>>> Not sure if it's right but probably we could reuse on fork an old anon_vma
>>> from the chain if it's already lost all vmas which points to it.
>>> For endlessly forking exploit this should work mostly like proposed patch
>>> which stops branching after some depth but without magic constant.
>>
>> Something like this. I leave proper comment for tomorrow.
>
> Hmm I'm not sure that will work as it is. If I understand it correctly, your
> patch can detect if the parent's anon_vma has no own references at the fork()
> time. But at the fork time, the parent is still alive, it only exits after the
> fork, right? So I guess it still has own references and the child will still
> allocate its new anon_vma, and the problem is not solved.

But it could reuse anon_vma from grandparent or older.
Count of anon_vmas in chain will be limited with count of alive processes.

I think it's better to describe this in terms of sets of anon_vma
instead hierarchy:
at clone vma inherits pages from parent together with set of anon_vma
which they belong.
For new pages it might allocate new anon_vma or reuse existing. After
my patch vma
will try to reuse anon_vma from that set which has no vmas which points to it.
As a result there will be no parent-child relation between anon_vma and
multiple pages might have equal (anon_vma, index) pair but I see no
problems here.

>
> So maybe we could detect that the own references dropped to zero when the parent
> does exit, and then change mapping of all relevant pages to the root anon_vma,
> destroy avc's of children and the anon_vma itself. But that sounds quite
> heavyweight :/
>
> Vlastimil
>
>>>
>>>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/