netdev - Re: [PATCH 00/17] Add memberof(), split some headers, and slightly simplify code

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 22 Nov 2021 12:36:32 +0000
From:   Jonathan Cameron <Jonathan.Cameron@...wei.com>
To:     Andy Shevchenko <andriy.shevchenko@...ux.intel.com>
CC:     "Alejandro Colomar (man-pages)" <alx.manpages@...il.com>,
        Arnd Bergmann <arnd@...db.de>,
        LKML <linux-kernel@...r.kernel.org>,
        Ajit Khaparde <ajit.khaparde@...adcom.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Bjorn Andersson <bjorn.andersson@...aro.org>,
        Borislav Petkov <bp@...e.de>,
        Corey Minyard <cminyard@...sta.com>, Chris Mason <clm@...com>,
        "Christian Brauner" <christian.brauner@...ntu.com>,
        David Sterba <dsterba@...e.com>,
        Jani Nikula <jani.nikula@...ux.intel.com>,
        Jason Wang <jasowang@...hat.com>,
        Jitendra Bhivare <jitendra.bhivare@...adcom.com>,
        John Hubbard <jhubbard@...dia.com>,
        "John S . Gruber" <JohnSGruber@...il.com>,
        "Joonas Lahtinen" <joonas.lahtinen@...ux.intel.com>,
        Josef Bacik <josef@...icpanda.com>,
        Kees Cook <keescook@...omium.org>,
        Ketan Mukadam <ketan.mukadam@...adcom.com>,
        Len Brown <lenb@...nel.org>,
        "Michael S. Tsirkin" <mst@...hat.com>,
        Miguel Ojeda <ojeda@...nel.org>,
        Mike Rapoport <rppt@...ux.ibm.com>,
        Nick Desaulniers <ndesaulniers@...gle.com>,
        "Rafael J. Wysocki" <rafael@...nel.org>,
        Rasmus Villemoes <linux@...musvillemoes.dk>,
        Rodrigo Vivi <rodrigo.vivi@...el.com>,
        Russell King <linux@...linux.org.uk>,
        Somnath Kotur <somnath.kotur@...adcom.com>,
        Sriharsha Basavapatna <sriharsha.basavapatna@...adcom.com>,
        Subbu Seetharaman <subbu.seetharaman@...adcom.com>,
        Intel Graphics <intel-gfx@...ts.freedesktop.org>,
        ACPI Devel Maling List <linux-acpi@...r.kernel.org>,
        Linux ARM <linux-arm-kernel@...ts.infradead.org>,
        linux-btrfs <linux-btrfs@...r.kernel.org>,
        linux-scsi <linux-scsi@...r.kernel.org>,
        Networking <netdev@...r.kernel.org>,
        "open list:DRM DRIVER FOR QEMU'S CIRRUS DEVICE" 
        <virtualization@...ts.linux-foundation.org>
Subject: Re: [PATCH 00/17] Add memberof(), split some headers, and slightly
 simplify code

On Fri, 19 Nov 2021 18:35:26 +0200
Andy Shevchenko <andriy.shevchenko@...ux.intel.com> wrote:

> On Fri, Nov 19, 2021 at 05:22:48PM +0100, Alejandro Colomar (man-pages) wrote:
> > 
> > 
> > On 11/19/21 17:18, Arnd Bergmann wrote:  
> > > On Fri, Nov 19, 2021 at 5:10 PM Andy Shevchenko
> > > <andriy.shevchenko@...ux.intel.com> wrote:  
> > >> On Fri, Nov 19, 2021 at 04:57:46PM +0100, Arnd Bergmann wrote:  
> > >   
> > >>> The main problem with this approach is that as soon as you start
> > >>> actually reducing the unneeded indirect includes, you end up with
> > >>> countless .c files that no longer build because they are missing a
> > >>> direct include for something that was always included somewhere
> > >>> deep underneath, so I needed a second set of scripts to add
> > >>> direct includes to every .c file.  
> > >>
> > >> Can't it be done with cocci support?  
> > > 
> > > There are many ways of doing it, but they all tend to suffer from the
> > > problem of identifying which headers are actually needed based on
> > > the contents of a file, and also figuring out where to put the extra
> > > #include if there are complex #ifdefs.
> > > 
> > > For reference, see below for the naive pattern matching I tried.
> > > This is obviously incomplete and partially wrong.  
> > 
> > FYI, if you may not know the tool,
> > theres include-what-you-use(1) (a.k.a. iwyu(1))[1],
> > although it is still not mature,
> > and I'm helping improve it a bit.  
> 
> Yes, I know the tool, but it produces insanity. Jonathan (maintainer
> of IIO subsystem) actually found it useful after manual work applied.
> Perhaps you can chat with him about usage of it in the Linux kernel.

IIO drivers use a fairly limited subset of headers, so it wasn't implausible
to produce a mapping file to get to fairly sane results.

https://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git/commit/?h=iio-iwyu-cleanups&id=8bc7ff8d5385e89a5199e792fda18dbf2ca8f2e5

If we did head towards a general mapping file that 'more or less made sense'
then maybe this tool can be fairly useful kernel wide.

Typical patch that results in clean checks with that mapping file is:
https://git.kernel.org/pub/scm/linux/kernel/git/jic23/iio.git/commit/?h=iio-iwyu-cleanups&id=0eff2dd097add84c464710003c3bc9929f646427

There are always going to be questions of how many of the really low level
direct includes make sense though which was the stumbling block.

It is nice to get rid of the pointless includes though for things that due
to refactors etc are no longer used in a file, or were cut and paste from
another driver.

I've paused efforts on this front for now given series like this one can
have significant impact and it seems to be an active area at the moment.
Might revisit later this cycle.

Jonathan

> 
> > If I understood better the kernel Makefiles,
> > I'd try it.
> > 
> > You can try it yourselves.
> > I still can't use it for my own code,
> > since it has a lot of false positives.  
> 
> > [1]: <https://include-what-you-use.org/>  
>