linux-kernel - Re: [10 PATCHES] inline functions to avoid stack overflow

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <Pine.LNX.4.64.0807031708280.20244@engineering.redhat.com>
Date:	Thu, 3 Jul 2008 17:12:18 -0400 (EDT)
From:	Mikulas Patocka <mpatocka@...hat.com>
To:	David Miller <davem@...emloft.net>
cc:	helge.hafting@...el.hist.no, sparclinux@...r.kernel.org,
	linux-kernel@...r.kernel.org, gcc@....gnu.org
Subject: Re: [10 PATCHES] inline functions to avoid stack overflow

On Tue, 1 Jul 2008, David Miller wrote:

> From: Mikulas Patocka <mpatocka@...hat.com>
> Date: Wed, 2 Jul 2008 00:39:35 -0400 (EDT)
>
>> The ABI is very vague about it. The V9 ABI just displays that 6-word space
>> in a figure bug doesn't say anything about it's usage. The V8 ABI just
>> says that "the function may write incoming arguments there". If it may
>> write anything other, it is unknown --- probably yes, but it is not said
>> in the document.
>>
>> The document nicely specifies who owns which registers, but doesn't say
>> that about the stack space :-(
>
> Actually, I know for a fact that you have to have those slots there.
>
> A long time ago in the sparc64 kernel, in the trap entry code, I tried
> only giving 128 bytes of stack frame as the trap entry called into C
> code.  And it did not work, I had to put the 6 slots there.

The bad thing is that gcc can't use those slots optimally. If you have for 
example:

void f(int *x)
{
}

void g()
{
 	int a;
 	f(&a);
}

void h()
{
 	g();
}

Then the variable "a" can't be placed into one of the 6 implicit slots for 
g->f call (beacuse "f" may overwrite that slot). But "a" could be placed 
into one of those 6 slots that "h" allocates for "g" (because these slots 
are owned by "g"). But it isn't --- additional place is allocated for "a" 
:-/

Mikulas
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/