linux-kernel - RE: violating function pointer signature

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <dade033f8ffa4b5fb01627bee0d55825@AcuMS.aculab.com>
Date:   Wed, 18 Nov 2020 16:19:09 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     'Mathieu Desnoyers' <mathieu.desnoyers@...icios.com>,
        rostedt <rostedt@...dmis.org>
CC:     Peter Zijlstra <peterz@...radead.org>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        Matt Mullins <mmullins@...x.us>,
        Ingo Molnar <mingo@...hat.com>,
        Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Andrii Nakryiko <andriin@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...omium.org>,
        netdev <netdev@...r.kernel.org>, bpf <bpf@...r.kernel.org>,
        Kees Cook <keescook@...omium.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        linux-toolchains <linux-toolchains@...r.kernel.org>
Subject: RE: violating function pointer signature

From: Mathieu Desnoyers
> Sent: 18 November 2020 16:01
...
> > If it is already done elsewhere in the kernel, then I will call this
> > precedence, and keep the original version.
> 
> It works for me. Bonus points if you can document in a comment that this
> trick depends on the cdecl calling convention.

It has nothing to do with 'cdecl' - which IIRC is a microsoft term.

Historically C just pushed arguments on the stack (no prototypes)
The calling code knew nothing about the called code or whether a
function might expect to have a variable number of arguments.
To stop this going horribly wrong the stack is tidied up by the caller.

PASCAL (which doesn't really support linking!) didn't support
variable argument lists and would get the called code to remove
the arguments (which is why x86 has a 'ret n' instruction).
In principle this generates smaller/faster code and many of the
32bit windows functions use it - probably due to turbo-pascal).

Modern calling conventions tend to pass some arguments in registers.
All the ones that get used (by default) on linux will get the
caller to tidy the stack.
Although some may use a simpler calling convention for varargs functions.

So a common 'return constant' function can be called from any call site.
But it you actually call a real function (that looks at the arguments)
you better have a matching prototype.
(eg cast the function pointer back to the correct one before the call.)

There are calling conventions where pointer and integer parameters
and results are passed in different registers.
The usual definition of ioctl() is typically broken.

	David

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)