linux-kernel - Re: Stack size protection broken on ppc64

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Sat, 6 Feb 2010 15:20:38 +1100
From:	Anton Blanchard <anton@...ba.org>
To:	Michael Neuling <mikey@...ling.org>
Cc:	linux-kernel@...r.kernel.org, linuxppc-dev@...abs.org,
	Serge Hallyn <serue@...ibm.com>,
	WANG Cong <xiyou.wangcong@...il.com>,
	Paul Mackerras <paulus@...ba.org>, benh@...nel.crashing.org,
	miltonm@....com, Andries.Brouwer@....nl
Subject: Re: Stack size protection broken on ppc64


Hi,

> On recent ppc64 kernels, limiting the stack (using 'ulimit -s blah') is
> now more restrictive than it was before.  On 2.6.31 with 4k pages I
> could run 'ulimit -s 16; /usr/bin/test' without a problem.  Now with
> mainline, even 'ulimit -s 64; /usr/bin/test' gets killed.
> 
> Using 64k pages is even worse.  I can't even run '/bin/ls' with a 1MB
> stack (ulimit -s 1024; /bin/ls).  Hence, it seems new kernels are too
> restrictive, rather than the old kernels being too liberal.

It looks like this is causing it:

#define EXTRA_STACK_VM_PAGES    20      /* random */

...

#ifdef CONFIG_STACK_GROWSUP
        stack_base = vma->vm_end + EXTRA_STACK_VM_PAGES * PAGE_SIZE;
#else
        stack_base = vma->vm_start - EXTRA_STACK_VM_PAGES * PAGE_SIZE;
#endif

Which got added back in 2005 in a memory overcommit patch. It only took 5
years for us to go back and review that random setting :)

The comment from Andries explains the purpose:

    (1) It reserves a reasonable amount of virtual stack space (amount
        randomly chosen, no guarantees given) when the process is started, so
        that the common utilities will not be killed by segfault on stack
        extension.

This explains why 64kB is much worse. The extra stack reserve should be in kB
and we also need to be careful not to ask for more than our rlimit.

Anton

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/