linux-kernel - Re: [v2 2/2] kbuild: copy scripts/atomic/atomic-*.h to include/generated/atomic-*.h

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <YoYK2pQ3NSN8tgdd@FVFF77S0Q05N>
Date:   Thu, 19 May 2022 10:16:10 +0100
From:   Mark Rutland <mark.rutland@....com>
To:     Masahiro Yamada <masahiroy@...nel.org>
Cc:     Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Vincent Mailhol <mailhol.vincent@...adoo.fr>,
        Will Deacon <will@...nel.org>,
        Peter Zijlstra <peterz@...radead.org>
Subject: Re: [v2 2/2] kbuild: copy scripts/atomic/atomic-*.h to
 include/generated/atomic-*.h

On Mon, May 16, 2022 at 06:30:41PM +0900, Masahiro Yamada wrote:
> On Mon, May 16, 2022 at 5:43 PM Mark Rutland <mark.rutland@....com> wrote:
> >
> > [adding the atomics maintainers to Cc]
> >
> > On Sat, May 14, 2022 at 01:02:07PM +0900, Masahiro Yamada wrote:
> > > include/linux/atomic/*.h are generated by scripts/atomic/gen-atomics.sh.
> > >
> > > To ensure they are not edited, scripts/atomic/check-atomics.sh checks
> > > sha1sum every time.
> > >
> > > This commit moves include/linux/atomic/*.h to scripts/atomic/*.h_shipped,
> > > which are copied to include/generated/ at build time:
> > >
> > >   COPY    include/generated/atomic-instrumented.h
> > >   COPY    include/generated/atomic-long.h
> > >   COPY    include/generated/atomic-arch-fallback.h
> >
> > FWIW, moving these and copying them at build time is fine by me, given that
> > better indicates that these are generated.
> >
> > > I dropped the sha1sum checks. I hope nobody would try to directly modify
> > > *_shipped files.
> >
> > I'd prefer to keep the sha1sum check, because we're worried that people *will*
> > modify them, and that won't be noticed until the next time they're regenerated.
> 
> OK, but is there any reason to embed the checksum to the headers?
> 
> Generally, you can have *.sha1 file to collect the checksums,
> and use the "sha1sum --check" command.

TBH, I have no preference either way; splitting it out is fine by me.

The only thing that needs to be ensured is that if the sha1sum tool doesn't
exist we don't complain about that.

[...]

> > > I guess it can be improved because it is just simple text processing.
> > > Then, Kbuild can execute it at build time.
> >
> > It is in theory possible to make them much faster, yes. The actual slowness of
> > the scripting is largely due to using sub-shells resulting in a load of
> > fork+exec, which could be avoided if using a real scripting language.
> >
> > Practically speaking though, we're rather limited in what we can rely upon
> > being available. We chose to use POSIX shell as a lowest-common-demoninator,
> > and anything that'd be *nice* to use isn't going to always be available,
> > meaning that even if we make it faster we can't necessarily build it all the
> > time anyway.
> 
> Kernel builds already rely on Perl.

I didn't realise that we always relied upon perl -- I know that there's
recordmcount.pl, but I didn't think all targets used that, and I thought we
couldn't rely upon perl when we originally wrote the atomci scripting.

If we can rely upom perl being available then I'm not averse to writing the
scripting in perl; I'm just not immediately sure about how to handle the
fallback templates.

Thanks,
Mark.