linux-kernel - Re: [PATCH] Documentation: dev-tools: Add a section for static analysis tools

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAGS_qxpjj1W54BM7v2Cszne4nh5kUXZt89Dq-5nO3nD7RWhsRQ@mail.gmail.com>
Date:   Mon, 28 Mar 2022 17:37:28 -0500
From:   Daniel Latypov <dlatypov@...gle.com>
To:     Marcelo Schmitt <marcelo.schmitt1@...il.com>
Cc:     corbet@....net, mchehab+huawei@...nel.org, davidgow@...gle.com,
        linux-doc@...r.kernel.org, linux-sparse@...r.kernel.org,
        cocci@...ia.fr, smatch@...r.kernel.org,
        linux-kernel@...r.kernel.org, skhan@...uxfoundation.org
Subject: Re: [PATCH] Documentation: dev-tools: Add a section for static
 analysis tools

On Mon, Mar 28, 2022 at 4:20 PM Marcelo Schmitt
<marcelo.schmitt1@...il.com> wrote:
>
> Complement the Kernel Testing Guide documentation page by adding a
> section about static analysis tools.
>
> Signed-off-by: Marcelo Schmitt <marcelo.schmitt1@...il.com>


> ---
> Hey everyone,
>
> I think this patch can be a good addition to the documentation as
> discussed in the thread for the testing guide documentation page:
> Link: https://lore.kernel.org/linux-doc/CABVgOS=2iYtqTVdxwH=mcFpcSuLP4cpJ4s6PKP4Gc-SH6jidgQ@mail.gmail.com/
>
> If you think it would be worth it, I can try making something more

Nice!
This is definitely worth including, but I'm out of my depth here.
Some ideas below in case they're helpful.

Acked-by: Daniel Latypov <dlatypov@...gle.com>

> elaborated. Maybe provide some guidance on when to use each tool.
> I've been studying how Linux device drivers are tested.
> Here's a post I wrote talking about some testing tools.
> Link: https://marcelosc.gitlab.io/how-is-linux-tested/
>
> Best regards,
> Marcelo
>
>  Documentation/dev-tools/testing-overview.rst | 29 ++++++++++++++++++++
>  1 file changed, 29 insertions(+)
>
> diff --git a/Documentation/dev-tools/testing-overview.rst b/Documentation/dev-tools/testing-overview.rst
> index 65feb81edb14..b00511109a9d 100644
> --- a/Documentation/dev-tools/testing-overview.rst
> +++ b/Documentation/dev-tools/testing-overview.rst
> @@ -115,3 +115,32 @@ that none of these errors are occurring during the test.
>  Some of these tools integrate with KUnit or kselftest and will
>  automatically fail tests if an issue is detected.
>
> +Static Analysis Tools
> +======================
> +
> +In addition to testing a running kernel, one may also scout for bugs by
> +analyzing the source code semantics. Three tools are well known for serving this
> +purpose.
> +
> +Sparse can help test the kernel by performing type-checking, lock checking,
> +value range checking, in addition to reporting various errors and warnings while
> +examining the code. See the Documentation/dev-tools/sparse.rst documentation
> +page for details on how to use it.
> +
> +Smatch extends Sparse and provides additional checks for programming logic

Coming from a place of ignorance, when should I use sparse vs smatch?
Is there an existing consensus on this, or is that a controversial question?

I assume other newcomers like me will have the same question and be
unsure if they should spend time learning about sparse.
In general, I think the main point of this page is to explain both
what the tools do and give an idea of _why_ someone might want to
consider using them.

> +mistakes such as missing breaks in switch statements, unused return values on
> +error checking, forgetting to set an error code in the return of an error path,
> +etc. Smatch also has tests against more serious issues such as integer
> +overflows, null pointer dereferences, and memory leaks. See the project page at
> +http://smatch.sourceforge.net/.
> +
> +We also have Coccinelle as an option within static analyzers. Coccinelle is
> +often used to aid collateral evolution of source code, but it can also help to
> +avoid certain bugs that have been expressed semantically. The types of tests
> +available include API tests, tests for correct usage of kernel iterators, checks
> +for the soundness of free operations, analysis of locking behavior, and further
> +tests known to help keep consistent kernel usage. See the
> +Documentation/dev-tools/coccinelle.rst documentation page for details.
> +
> +These static analysis tools support running tests on the whole source tree or
> +over a specific file or directory.

Personal preference:
I'd personally put this up before we go into details about each tool.

E.g. perhaps like

In addition to testing a running kernel, one can also analyze kernel
source code directly (the whole tree or specific files) using **static
analysis** tools.
Some tools commonly used in the kernel are ...

> --
> 2.35.1
>