linux-kernel - Re: build reproducibility

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNAS2H9n9NSpmMk_AkX1N9JRa_5Rk9Sgeguq_9uepzd1f5w@mail.gmail.com>
Date:   Mon, 18 Oct 2021 11:26:10 +0900
From:   Masahiro Yamada <masahiroy@...nel.org>
To:     Julia Lawall <julia.lawall@...ia.fr>
Cc:     Randy Dunlap <rdunlap@...radead.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>
Subject: Re: build reproducibility

On Mon, Oct 18, 2021 at 3:42 AM Julia Lawall <julia.lawall@...ia.fr> wrote:
>
>
>
> On Sun, 17 Oct 2021, Randy Dunlap wrote:
>
> > On 10/17/21 11:12 AM, Julia Lawall wrote:
> > > Hello,
> > >
> > > If I do the following:
> > >
> > > git clean -dfx
> > > cp saved_config .config
> > > make olddefconfig && make && make modules_install && make install
> > >
> > > Should I always end up with the same kernel, regardless of the kernel that
> > > is currently running on the machine?
> > >
> > > I see a large performance difference between Linux 5.10 and all versions
> > > afterwards for a particular benchmark.  I am unable to bisect the problem
> > > eg between 5.10 and 5.11, because as soon as I come to a kernel that gives
> > > the bad performance, all of the kernels that I generate subsequently in
> > > the bisecting process (using the above commands) also have the bad
> > > performance.
> > >
> > > It could of course be that I have completely misinterpreted the problem,
> > > and it has nothing to do with the kernel.  But I have tested the program a
> > > lot when only working on variants of Linux 5.9.  I only start to have
> > > problems when I use versions >= 5.11.
> >
> > Hi,
> >
> > My "guess" is that this has something to do with the build
> > reusing some current file(s) that need to be rebuilt.
> > I.e., adding a "make clean" or "make proper" might be needed.
>
> This was my guess too.  But I have the git clean -dfx.  I did a comparison
> with make distclean and this does a little more (mostly some files in
> tools).
>
> thanks,
> julia
>


'git clean -dfx' is a very hard cleaning.
So, you are doing a full build in every step of bisecting.

I have no idea to explain the symptom you observed:
 "as soon as I come to a kernel that gives
 the bad performance, all of the kernels that I generate subsequently in
 the bisecting process"


If you desire perfect reproducibility, you can check
   Documentation/kbuild/reproducible-builds.rst
But, I doubt slight differences such as timestamps
can explain the large performance difference.


If you are chasing the performance issue,
commit cf536e185869d4815 said
CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_*
might be useful to eliminate the possibility
of code alignment.


Otherwise, I have no more idea...






-- 
Best Regards
Masahiro Yamada