linux-kernel - Re: [PATCH v5 1/2] check-uapi: Introduce check-uapi.sh

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNAQQmoyUx+0Jk3c7iqY20KokrHEOPwHNb2doZOOA8RWBDA@mail.gmail.com>
Date:   Mon, 10 Apr 2023 19:03:05 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     John Moon <quic_johmoo@...cinc.com>
Cc:     Nathan Chancellor <nathan@...nel.org>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        Nicolas Schier <nicolas@...sle.eu>,
        linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        linux-arm-kernel@...ts.infradead.org,
        linux-arm-msm@...r.kernel.org,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Randy Dunlap <rdunlap@...radead.org>,
        Arnd Bergmann <arnd@...db.de>,
        Bjorn Andersson <andersson@...nel.org>,
        Todd Kjos <tkjos@...gle.com>,
        Matthias Maennich <maennich@...gle.com>,
        Giuliano Procida <gprocida@...gle.com>,
        kernel-team@...roid.com, libabigail@...rceware.org,
        Jordan Crouse <jorcrous@...zon.com>,
        Trilok Soni <quic_tsoni@...cinc.com>,
        Satya Durga Srinivasu Prabhala <quic_satyap@...cinc.com>,
        Elliot Berman <quic_eberman@...cinc.com>,
        Guru Das Srinagesh <quic_gurus@...cinc.com>
Subject: Re: [PATCH v5 1/2] check-uapi: Introduce check-uapi.sh

On Sat, Apr 8, 2023 at 5:35 AM John Moon <quic_johmoo@...cinc.com> wrote:
>
> While the kernel community has been good at maintaining backwards
> compatibility with kernel UAPIs, it would be helpful to have a tool
> to check if a commit introduces changes that break backwards
> compatibility.
>
> To that end, introduce check-uapi.sh: a simple shell script that
> checks for changes to UAPI headers using libabigail.
>
> libabigail is "a framework which aims at helping developers and
> software distributors to spot some ABI-related issues like interface
> incompatibility in ELF shared libraries by performing a static
> analysis of the ELF binaries at hand."
>
> The script uses one of libabigail's tools, "abidiff", to compile the
> changed header before and after the commit to detect any changes.
>
> abidiff "compares the ABI of two shared libraries in ELF format. It
> emits a meaningful report describing the differences between the two
> ABIs."
>
> The script also includes the ability to check the compatibility of
> all UAPI headers across commits. This allows developers to inspect
> the stability of the UAPIs over time.
>
> Signed-off-by: John Moon <quic_johmoo@...cinc.com>

BTW, is there anybody (except the submitters) who loves this tool?
(or anybody who has ever evaluated this?)

I am the only person who pointed out something.

I put some more comments from users' point of view.

[1] The number of lines of the error log.

According to this tool, it looks like we broke a lot of UAPI
headers in the previous MW (between v6.2 and v6.3-rc1).

Not only in the previous MW, we always broke so many UAPI
headers, this script says.

The following command outputs 2380 lines.

$ ./scripts/check-uapi.sh -q -b v6.3-rc1 -p v6.2 2>&1  | wc
   2380    9438  104953

I do not know how many are real breakages, and
how many are false positives.

Anyway, I will not check the 2380 lines of the error log.

And, please note that the help message explains '-q' is the *quiet* mode.
It is the opposite. This is super noisy.

[2] Be careful!

While testing the patch submissions of this,
I messed up my repository multiple times.

The script takes some time because it builds many objects
internally.

However, once this script starts running, you must not hit Ctrl-C.
If you do it, your repository will be sprinkled with a ton
of untracked files.

Apply this patch, and run "./scripts/check-uapi.sh -p v6.0"
and hit Ctrl-C.

Repeat it a couple of times, and "git status" will show you
something horrible.

You will never know when git is checking out a commit
because this script hides it by 'git checkout --quiet'.

So, this tool should show a caveat at least.

'git checkout' should not be hidden, or
maybe a caveat message should be shown.

CAVEAT
This tool runs 'git checkout' a couple of times internally.
If you interrupt it, your worktree might be messed up.

--
Best Regards
Masahiro Yamada