linux-kernel - Re: [BUGFIX][PATCH] fixing placement of register stack under ulimit -s

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20070316073047.8c92bcd1.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Fri, 16 Mar 2007 07:30:47 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
Cc:	dmosberger@...il.com, linux-ia64@...r.kernel.org,
	tony.luck@...el.com, linux-kernel@...r.kernel.org
Subject: Re: [BUGFIX][PATCH] fixing placement of register stack under ulimit
 -s

On Fri, 16 Mar 2007 06:20:47 +0900
KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com> wrote:

> On Thu, 15 Mar 2007 09:57:28 -0600
> "David Mosberger-Tang" <dmosberger@...il.com> wrote:
> 
> > But aren't you going to be limited to less than a page worth of
> > register-backing store even with your patch applied because the
> > backing store will end up overflowing the memory stack?
> > 
> 
> I think pthread's stack, which is created by malloc, is also shared
> among register-stack and memory-stack. 
> (glibc's pthread's stack is limited by ulimit, too.)
> 
> So, it seems stack_size_limit = register_stack_limit + memory_stack_limit
> is a consistent way. I'm sorry if I don't catch your point.
> 
BTW, what way do you recommened to fix this register-stack/memory-stack upside
down problem ?

Plan A) just handle upside-down case in page fault handler.
	This means ulimit -s limitation will limit amount of memory-stack and
	register stack independently.
Plan B) handle upside-down case in page fault handler and add modify 
 	acct_stack_growth() to be able to handle the limitation of sum
	of separated vmas.(vma for reg stack and mem stack). 
Plan C) don't allow this upside down as this patch. but change calculation of rbs_top.

Note:
To see the problem which my patch want to fix run following code under ulimit -s.

==
void eat_stack(int num) {
	printf("%d\n", num);
	eat_stack(num - 1);
}

int main (void) {
	eat_stack(10000);
}
==

-- Kame

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/