linux-kernel - Re: Subject: Re: [PATCH v3] kbuild: add support for zstd compressed modules

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+icZUWbYd3z-+FDoXGx5UQcY4R1BuBn5V=o0d06=XADOZD8gQ@mail.gmail.com>
Date:   Fri, 9 Apr 2021 13:31:07 +0200
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Piotr Gorski <lucjan.lucjanov@...il.com>
Cc:     linux-kbuild@...r.kernel.org, linux-kernel@...r.kernel.org,
        masahiroy@...nel.org, oleksandr@...alenko.name
Subject: Re: Subject: Re: [PATCH v3] kbuild: add support for zstd compressed modules

On Fri, Apr 9, 2021 at 1:10 PM Piotr Gorski <lucjan.lucjanov@...il.com> wrote:
>
> I originally posted the patch in a different form [1] even before Masahiro's changes.
> I've been testing this solution since December last year and posted it in March this year,
> after I made sure everything was working fine. This patch was tested by Oleksandr and he also didn't report any objections. [2]
>
> Masahiro notified me about the planned changes [3] and asked me to resend this patch, adjusted to those changes, which I did.
>
> My current logs:
>
> lucjan@...hlinux ~ $ zgrep CONFIG_DEBUG_INFO /proc/config.gz
> CONFIG_DEBUG_INFO=y
> # CONFIG_DEBUG_INFO_REDUCED is not set
> # CONFIG_DEBUG_INFO_COMPRESSED is not set
> # CONFIG_DEBUG_INFO_SPLIT is not set
> CONFIG_DEBUG_INFO_DWARF4=y
> CONFIG_DEBUG_INFO_BTF=y
> CONFIG_DEBUG_INFO_BTF_MODULES=y
> lucjan@...hlinux ~ $ zgrep CONFIG_MODULE_COMPRESS_ZSTD /proc/config.gz
> CONFIG_MODULE_COMPRESS_ZSTD=y
> CONFIG_MODULE_COMPRESS_ZSTD_LEVEL=19
>
> Pay no attention to CONFIG_MODULE_COMPRESS_ZSTD_LEVEL as this is not in the upstream, it's an additional patch I use.
>
> The only difference - I don't use clang. Maybe those who use will comment on this.
> I relied on the opinions of Oleksander and a dozen other users who reported no errors in using zstd module compression.
>
> [1] https://marc.info/?l=linux-kbuild&m=161710402402989&w=2
>
> [2] https://marc.info/?l=linux-kbuild&m=161710503403517&w=2
>
> [3] https://marc.info/?l=linux-kbuild&m=161780602730829&w=2

I am a big fan of ZSTD and have it as default in all available Linux
Kconfigs and Debian's initramfs-tools.
So, I am highly interested in getting this fixed.

Unfortunately, I have thrown away my yesterday's Clang-LTO build and
switched to Clang-CFI with builddeb - should do handle the same way.

I see three iwlwifi.ko (as an example):

$ LC_ALL=C ll drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
-rw-r--r-- 1 dileks dileks 8.2M Apr  9 11:07
drivers/net/wireless/intel/iwlwifi/iwlwifi.ko

$ file drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
drivers/net/wireless/intel/iwlwifi/iwlwifi.ko: ELF 64-bit LSB
relocatable, x86-64, version 1 (SYSV),
BuildID[sha1]=78d593f4fd2b8efe81caeb8f1ea729107a33e244, with
debug_info, not stripped

That iwlwifi.ko with debug-info is optimized when moving to
debian/linux-image-dbg directory:

$ LC_ALL=C ll debian/linux-image-dbg/usr/lib/debug/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
-rw-r--r-- 1 dileks dileks 7.9M Apr  9 11:18
debian/linux-image-dbg/usr/lib/debug/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko

$ file debian/linux-image-dbg/usr/lib/debug/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
debian/linux-image-dbg/usr/lib/debug/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko:
ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV),
BuildID[sha1]=78d593f4fd2b8efe81caeb8f1ea729107a33e244, with
debug_info, not stripped

And think it's shrunk down and included debian/linux-image directory:

$ LC_ALL=C ll debian/linux-image/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
-rw-r--r-- 1 dileks dileks 694K Apr  9 11:18
debian/linux-image/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko

$ file debian/linux-image/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko
debian/linux-image/lib/modules/5.12.0-rc6-5-amd64-clang12-cfi/kernel/drivers/net/wireless/intel/iwlwifi/iwlwifi.ko:
ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV),
BuildID[sha1]=78d593f4fd2b8efe81caeb8f1ea729107a33e244, not stripped

I speculate both iwlwifi.ko below debian directory should be ZSTD-compressed.
Fact is the one with debug-info is done correctly.
Might be builddeb script needs a special treatment.

- Sedat -