full-disclosure - Local variable memory allocation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <1063218823.b0a63f80MXB285@bham.ac.uk>
From: MXB285 at bham.ac.uk (M Bealby)
Subject: Local variable memory allocation

I am trying to learn about the security vulnerabilities created by buffer overflows. I am trying to work through Aleph1's excellent paper 'Smashing the stack for fun and profit', but when I compile the examples, I get different results. I know the results will not be exactly the same, but I get the same difference on two different machines I have tried it upon. Because of this I guess it's not an error, but something else. The machines are a Pentium-mmx (laptop) with gcc 3.0 and 2.95, and a Pentium-4 with gcc 3.2.2 and 2.96.

This is the code I am compiling:

--- START CODE ---

void function(int a, int b, int c) {
        char buffer[VALUE];
}

void main() {
        function(1, 2, 3);
}

--- END CODE ---

As you can see it's a variation of 'example1.c' from Aleph1's paper.

Different values for 'VALUE' were tried and compiled on both machines with both compilers, with and without explicitly stating no optimization (-O0).

When VALUE=1, this is what I get from gdb:

--- START GDB ---

(gdb) disassemble function
Dump of assembler code for function function:
0x080482f4 <function+0>:        push   %ebp
0x080482f5 <function+1>:        mov    %esp,%ebp
0x080482f7 <function+3>:        sub    $0x4,%esp
0x080482fa <function+6>:        leave
0x080482fb <function+7>:        ret
End of assembler dump.
(gdb)

--- END GDB ---

As far as I know this is allocating 4 bytes for the local variables (as expected on a 32 bit machine). This is what I expected.

When VALUE=4, I get this disassembly:

--- START GDB ---

(gdb) disassemble function
Dump of assembler code for function function:
0x080482f4 <function+0>:        push   %ebp
0x080482f5 <function+1>:        mov    %esp,%ebp
0x080482f7 <function+3>:        sub    $0x4,%esp
0x080482fa <function+6>:        leave
0x080482fb <function+7>:        ret
End of assembler dump.
(gdb)

--- END GDB ---

This is the same as above, and also what I was expecting.

When VALUE=5, this is what gdb gives me:

--- START GDB ---

(gdb) disassemble function
Dump of assembler code for function function:
0x080482f4 <function+0>:        push   %ebp
0x080482f5 <function+1>:        mov    %esp,%ebp
0x080482f7 <function+3>:        sub    $0x18,%esp
0x080482fa <function+6>:        leave
0x080482fb <function+7>:        ret
End of assembler dump.
(gdb)

--- END GDB ---

For some reason it is allocating 24 (0x18) bytes for local variables. This is what I do not understand as I would have expected it to allocate 8 bytes (2 words).
However, when VALUE=8, I get the expected result of an allocation of 8 bytes (2 words).

Could someone please shed some light on this for me please, as this is happening on two different computers, running different Linux distributions with both a 2.9 and a 3.x series gcc. I'm sure that I am doing something wrong, but I do not know where.

Thanks,
Martin