linux-kernel - Re: [PATCH] kheaders: substituting --sort in archive creation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191014134037.GA79684@google.com>
Date:   Mon, 14 Oct 2019 14:40:37 +0100
From:   Quentin Perret <qperret@...gle.com>
To:     Dmitry Goldin <dgoldin@...tonmail.ch>
Cc:     Masahiro Yamada <yamada.masahiro@...ionext.com>,
        Andreas Schwab <schwab@...ux-m68k.org>,
        Greg KH <gregkh@...uxfoundation.org>,
        "linux-kernel\\\\\\\\@vger.kernel.org" <linux-kernel@...r.kernel.org>,
        "joel\\\\\\\\@joelfernandes.org" <joel@...lfernandes.org>,
        Ben Hutchings <ben@...adent.org.uk>, adelva@...gle.com,
        natechancellor@...il.com, maennich@...gle.com
Subject: Re: [PATCH] kheaders: substituting --sort in archive creation

Hi Dmitry,

On Wednesday 09 Oct 2019 at 13:42:14 (+0000), Dmitry Goldin wrote:
> From: Dmitry Goldin <dgoldin+lkml@...tonmail.ch>
> 
> The option --sort=ORDER was only introduced in tar 1.28 (2014), which
> is rather new and might not be available in some setups.
> 
> This patch tries to replicate the previous behaviour as closely as possible
> to fix the kheaders build for older environments. It does not produce identical
> archives compared to the previous version due to minor sorting
> differences but produces reproducible results itself in my tests.
> 
> Signed-off-by: Dmitry Goldin <dgoldin+lkml@...tonmail.ch>
> ---
>  kernel/gen_kheaders.sh | 11 +++++++----
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh
> index aff79e461fc9..5a0fc0b0403a 100755
> --- a/kernel/gen_kheaders.sh
> +++ b/kernel/gen_kheaders.sh
> @@ -71,10 +71,13 @@ done | cpio --quiet -pd $cpio_dir >/dev/null 2>&1
>  find $cpio_dir -type f -print0 |
>  	xargs -0 -P8 -n1 perl -pi -e 'BEGIN {undef $/;}; s/\/\*((?!SPDX).)*?\*\///smg;'
> 
> -# Create archive and try to normalize metadata for reproducibility
> -tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> -    --owner=0 --group=0 --sort=name --numeric-owner \
> -    -Jcf $tarfile -C $cpio_dir/ . > /dev/null
> +# Create archive and try to normalize metadata for reproducibility.
> +# For compatibility with older versions of tar, files are fed to tar
> +# pre-sorted, as --sort=name might not be available.
> +find $cpio_dir -printf "./%P\n" | LC_ALL=C sort | \
> +    tar "${KBUILD_BUILD_TIMESTAMP:+--mtime=$KBUILD_BUILD_TIMESTAMP}" \
> +    --owner=0 --group=0 --numeric-owner --no-recursion \
> +    -Jcf $tarfile -C $cpio_dir/ -T - > /dev/null
> 
>  echo "$src_files_md5" >  kernel/kheaders.md5
>  echo "$obj_files_md5" >> kernel/kheaders.md5
> --
> 2.23.0

FWIW:

  Tested-by: Quentin Perret <qperret@...gle.com>

It turns out this issue broke something in our CI, could this patch be
queued as a -rc4 fix ?

Thanks,
Quentin