linux-kernel - RE: detecting integer constant expressions in macros

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 21 Mar 2018 10:35:58 +0000
From:   David Laight <David.Laight@...LAB.COM>
To:     "'Uecker, Martin'" <Martin.Uecker@....uni-goettingen.de>,
        "torvalds@...ux-foundation.org" <torvalds@...ux-foundation.org>
CC:     "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: detecting integer constant expressions in macros

From: Uecker, Martin
> Sent: 21 March 2018 10:22
> Am Mittwoch, den 21.03.2018, 10:51 +0100 schrieb Martin Uecker:
> >
> > Am Dienstag, den 20.03.2018, 17:30 -0700 schrieb Linus Torvalds:
> > > On Tue, Mar 20, 2018 at 5:10 PM, Uecker, Martin
> > > <Martin.Uecker@....uni-goettingen.de> wrote:
> > >
> > > > But one could also use __builtin_types_compatible_p instead.
> > >
> > > That might be the right approach, even if I like how it only used
> > > standard C (although _disgusting_ standard C) without it apart from
> > > the small issue of sizeof(void)
> > >
> > > So something like
> > >
> > >   #define __is_constant(a) \
> > >         __builtin_types_compatible_p(int *, typeof(1 ? ((void*)((a) * 0l)) : (int*)1 ) )
> > >
> > > if I counted the parentheses right..
> >
> > This seems to work fine on all recent compilers. Sadly, it
> > produces false positives on 4.4.7 and earlier when
> > tested on godbolt.org
> >
> > Surprisingly, the MAX macro as defined below still seems
> > to do the right thing with respect to avoiding the VLA
> > even on the old compilers.
> >
> > I am probably missing something... or there are two
> > compiler bugs cancelling out, or the __builting_choose_expr
> > changes things.
> 
> Nevermind, of course it avoids the VLA if it produces a false
> positive and uses the simple version. So it is unsafe to use
> on very old compilers.

False positives with old compilers don't matter when max() is being used
for an on-stack array.
The compilations with a new compiler will detect real VLA, the old compiler
will generate valid code with a constant sized VLA.

OTOH these horrid:
	long buf[max(sizeof (struct foo), sizeof (struct bar)) + 7 / 8];
would be better replaced with:
	union buf { struct foo foo; struct bar bar; };

	David