linux-kernel - Re: [PATCH v5 0/2] Remove false-positive VLAs when using max()

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <38b6da49-1138-017e-7307-f39ff067d6d2@rasmusvillemoes.dk>
Date:   Sun, 18 Mar 2018 23:59:21 +0100
From:   Rasmus Villemoes <linux@...musvillemoes.dk>
To:     Linus Torvalds <torvalds@...ux-foundation.org>,
        Rasmus Villemoes <linux@...musvillemoes.dk>
Cc:     Kees Cook <keescook@...omium.org>,
        Al Viro <viro@...iv.linux.org.uk>,
        Florian Weimer <fweimer@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Randy Dunlap <rdunlap@...radead.org>,
        Miguel Ojeda <miguel.ojeda.sandonis@...il.com>,
        Ingo Molnar <mingo@...nel.org>,
        David Laight <David.Laight@...lab.com>,
        Ian Abbott <abbotti@....co.uk>,
        linux-input <linux-input@...r.kernel.org>,
        linux-btrfs <linux-btrfs@...r.kernel.org>,
        Network Development <netdev@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Kernel Hardening <kernel-hardening@...ts.openwall.com>
Subject: Re: [PATCH v5 0/2] Remove false-positive VLAs when using max()

On 2018-03-18 22:33, Linus Torvalds wrote:
> On Sun, Mar 18, 2018 at 2:13 PM, Rasmus Villemoes
> <linux@...musvillemoes.dk> wrote:
>> On 2018-03-17 19:52, Linus Torvalds wrote:
>>>
>>> Ok, so it really looks like that same "__builtin_constant_p() doesn't
>>> return a constant".
>>>
>>> Which is really odd, but there you have it.
>>
>> Not really. We do rely on builtin_constant_p not being folded too
>> quickly to a 0/1 answer, so that gcc still generates good code even if
>> the argument is only known to be constant at a late(r) optimization
>> stage (through inlining and all).
> 
> Hmm. That makes sense. It just doesn't work for our case when we
> really want to choose the expression based on side effects or not.
> 
>> So unlike types_compatible_p, which
>> can obviously be answered early during parsing, builtin_constant_p is
>> most of the time a yes/no/maybe/it's complicated thing.
> 
> The silly thing is, the thing we *really* want to know _is_ actually
> easily accessible during the early parsing, exactly like
> __builtin_compatible_p(): it's not really that we care about the
> expressions being constant, as much as the "can this have side
> effects" question.

OK, I missed where this was made about side effects of x and y, but I
suppose the idea was to use

  no_side_effects(x) && no_side_effects(y) ? trivial_max(x, y) :
old_max(x, y)

or the same thing spelled with b_c_e? Yes, I think that would work, if
we indeed had that way of checking an expression.

> We only really use __builtin_constant_p() as a (bad) approximation of
> that in this case, since it's the best we can do.

I don't think you should parenthesize bad, rather capitalize it. ({x++;
0;}) is constant in the eyes of __builtin_constant_p, but not
side-effect free. Sure, that's very contrived, but I can easily imagine
some max(f(foo), -1) call where f is sometimes an external function, but
for other .configs it's a static inline that always returns 0, but still
has some non-trivial side-effect before that. And this would all depend
on which optimizations gcc applies before it decides to evaluate
builtin_constant_p, so could be some fun debugging. Good thing that that
didn't work out...

> So the real use would be to say "can we use the simple direct macro
> that just happens to also fold into a nice integer constant
> expression" for __builtin_choose_expr().
> 
> I tried to find something like that, but it really doesn't exist, even
> though I would actually have expected it to be a somewhat common
> concern for macro writers: write a macro that works in any arbitrary
> environment.

Yeah, I think the closest is a five year old suggestion
(https://gcc.gnu.org/bugzilla/show_bug.cgi?id=57612) to add a
__builtin_assert_no_side_effects, that could be used in macros that for
some reason cannot be implemented without evaluating some argument
multiple times. It would be more useful to have the predicate version,
which one could always turn into a build bug version. But we have
neither, unfortunately.

Rasmus