linux-kernel - Re: [PATCH 0/3] kbuild: clang-tidy

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <CAKwvOdm3VTZ2QXXxf9pjM6n87UE=Lc-9Cx=V70sNsYGmHCb-hA@mail.gmail.com>
Date:   Wed, 12 Aug 2020 15:52:54 -0700
From:   Nick Desaulniers <ndesaulniers@...gle.com>
To:     Nathan Huckleberry <nhuck@...gle.com>
Cc:     Masahiro Yamada <masahiroy@...nel.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Tom Roeder <tmroeder@...gle.com>,
        clang-built-linux <clang-built-linux@...glegroups.com>,
        "David S. Miller" <davem@...emloft.net>,
        Mauro Carvalho Chehab <mchehab+huawei@...nel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        Rob Herring <robh@...nel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 0/3] kbuild: clang-tidy

On Wed, Aug 12, 2020 at 12:56 PM Nathan Huckleberry <nhuck@...gle.com> wrote:
>
> On Wed, Aug 12, 2020 at 12:40 PM Masahiro Yamada <masahiroy@...nel.org> wrote:
> >
> >
> > I improved gen_compile_commands.py in the first two patches,
> > then rebased Nathan's v7 [1] on top of them.
> > To save time, I modified the Makefile part.
> > No change for run-clang-tools.py
> >
> > I am not sure if the new directory, scripts/clang-tools/,
> > is worth creating only for 2 files, but I do not have
> > a strong opinion about it.
> >
> > "make clang-tidy" should work in-tree build,
> > out-of-tree build (O=), and external module build (M=).
> > Tests and reviews are appreciated.
> >
> > "make clang-tidy" worked for me.
> >
> > masahiro@...ar:~/workspace/linux-kbuild$ make -j24 CC=clang clang-tidy
> >   DESCEND  objtool
> >   CALL    scripts/atomic/check-atomics.sh
> >   CALL    scripts/checksyscalls.sh
> >   CHK     include/generated/compile.h
> >   GEN     compile_commands.json
> >   CHECK   compile_commands.json
> >
> > But "make clang-analyzer" just sprinkled the following error:
> >
> >   Error: no checks enabled.
> >   USAGE: clang-tidy [options] <source0> [... <sourceN>]

I wasn't able to reproduce Masahiro's reported failure, but seeing as
he has `GEN` for compile_commands.json and I have `CHK`, I wonder if
that's from a run when the series was still under development?

I can reproduce if I run:
$ clang-tidy '-checks='
so maybe was string quoting problem?

> >
> > I built clang-tidy from the latest source.
> > I had no idea how to make it work...
>
> How are you building clang-tidy? The clang static-analyzer may not
> have been built.
> I believe the static analyzer is built as a part of clang, not as a
> part of clang-tools-extra.
>
> I use this command to build.
> cmake -DCMAKE_BUILD_TYPE="release"
> -DLLVM_TARGETS_TO_BUILD="X86;AArch64;ARM;RISCV"
> -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;llvm-as"
> -DLLVM_ENABLE_LLD=1 -G "Ninja" ../llvm
>
> Adding clang to the list of -DLLVM_ENABLE_PROJECTS will build the
> static analyzer.
> -DCLANG_ENABLE_STATIC_ANALYZER=1 might also work, but I haven't tested it.
>
> I tested the patchset and both clang-tidy and clang-analyzer work for me.

If you rename clang-tidy in your build dir, and ensure you don't have
a `clang-tidy` in your $PATH (`which clang-tidy`), maybe there's more
we can do to politely inform the user they're missing a dependency to
execute the make target?  Not sure if we could could test that
clang-tidy supports the clang-analyzer-* checks.  Isn't there an
invocation that prints the supported checks? `clang-tidy '-checks=*'
--list-checks` is in my shell history.  Maybe grepping that and
informing the user how to fix the problem might solve a "papercut?"

If I remove clang-tidy with this series applied, I get (the failure is
obvious to me, but...):
```
$ make LLVM=1 -j71 clang-tidy
...
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
    return list(map(*args))
  File "./scripts/clang-tools/run-clang-tools.py", line 54, in run_analysis
    p = subprocess.run(["clang-tidy", "-p", args.path, checks, entry["file"]],
  File "/usr/lib/python3.8/subprocess.py", line 489, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/usr/lib/python3.8/subprocess.py", line 854, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.8/subprocess.py", line 1702, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'clang-tidy'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./scripts/clang-tools/run-clang-tools.py", line 74, in <module>
    main()
  File "./scripts/clang-tools/run-clang-tools.py", line 70, in main
    pool.map(run_analysis, datastore)
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
    raise self._value
FileNotFoundError: [Errno 2] No such file or directory: 'clang-tidy'
make: *** [Makefile:1861: clang-tidy] Error 1
```
$ clang-tidy '-checks=*' --list-checks | grep clang-analyzer | wc -l
111

And I'm not sure you can even build clang or clang-tidy but not the analyzer.

>
> >
> > [1] https://patchwork.kernel.org/patch/11687833/
> >
> >
> >
> > Masahiro Yamada (2):
> >   gen_compile_commands: parse only the first line of .*.cmd files
> >   gen_compile_commands: wire up build rule to Makefile
> >
> > Nathan Huckleberry (1):
> >   Makefile: Add clang-tidy and static analyzer support to makefile
> >
> >  MAINTAINERS                                 |   1 +
> >  Makefile                                    |  45 +++++-
> >  scripts/clang-tools/gen_compile_commands.py | 117 +++++++++++++++
> >  scripts/clang-tools/run-clang-tools.py      |  74 ++++++++++
> >  scripts/gen_compile_commands.py             | 151 --------------------
> >  5 files changed, 233 insertions(+), 155 deletions(-)
> >  create mode 100755 scripts/clang-tools/gen_compile_commands.py
> >  create mode 100755 scripts/clang-tools/run-clang-tools.py
> >  delete mode 100755 scripts/gen_compile_commands.py
> >
> > --
> > 2.25.1
> >



-- 
Thanks,
~Nick Desaulniers