linux-kernel - Re: sparse annotation for error types?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAHk-=wjQYjRusiGfXTywSg5xC8knhP6uesDfO3J=bgu5uevtyA@mail.gmail.com>
Date:   Sat, 5 Dec 2020 15:10:15 -0800
From:   Linus Torvalds <torvalds@...ux-foundation.org>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     Sparse Mailing-list <linux-sparse@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        edwin.peer@...adcom.com,
        Zhang Changzhong <zhangchangzhong@...wei.com>
Subject: Re: sparse annotation for error types?

On Sat, Dec 5, 2020 at 2:34 PM Jakub Kicinski <kuba@...nel.org> wrote:
>
> Am I the only one who thinks this would be a good idea?

I don't think it's likely to be very useful, because a very common
pattern is to not have that separate "return 0" in the middle, but
more along the lines of

        err = first_step(obj, 1);
        if (err)
                return err;

        if (some_check(obj)) {
                err = -EINVAL; /* need explicit error set! */
                goto err_undo_1s;
        }

        err = second_step(obj, param);
        if (err)
                goto err_undo_1s;

        err = third_step(obj, 0);

   err_undo_2s:
        second_undo(obj);
   err_undo_1s:
        first_undo(obj);
        return err;

iow, the "undo" parts are often done even for the success cases. This
is particularly true when those first steps are locking-related, and
the code always wants to unlock.

Sparse also doesn't really do any value analysis, so I suspect it
wouldn't be trivial to implement in sparse anyway.

Syntactically, I also think it's wrong to annotate the variable - I
think the place to annotate would be the return statement, and say
"must be negative" there. Kind of similar to having function arguments
annotated as "must not be NULL" (which sparse also doesn't do, but
some other checking tools do, and sparse can at least _parse_
"__nonnull" even if it ends up being ignored).

It's a bit similar to gcc's has a "returns_nonnull" function
attribute, but that one is not "per return", it's a global "this
function cannot return NULL" thing so that callers can then be
optimized and NULL checks removed. So it's very similar to the
"argument is non-null" in that it's for warnings at the *caller*, not
the function itself.

So if we want sparse support for this, I'd suggest something more akin
to a smarter compile-time assert, IOW more like having a

        compile_time_assert(err < 0);
        return err;

and then sparse (or any other checker) could warn when there's a path
that results in "err" not being negative.

Having some kind of smarter compile-time assert could be useful in
general, but as mentioned, sparse doesn't really do value range
propagation right now, so..

Luc, any reactions?

          Linus