lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:   Wed, 9 Nov 2022 00:41:15 +0100
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Nick Desaulniers <ndesaulniers@...gle.com>
Cc:     Masahiro Yamada <masahiroy@...nel.org>,
        Fangrui Song <maskray@...gle.com>,
        Sedat Dilek <sedat.dilek@....com>,
        Nathan Chancellor <nathan@...nel.org>,
        Michal Marek <michal.lkml@...kovi.net>,
        Nick Terrell <terrelln@...com>, Tom Rix <trix@...hat.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "Peter Zijlstra (Intel)" <peterz@...radead.org>,
        Kees Cook <keescook@...omium.org>,
        Josh Poimboeuf <jpoimboe@...nel.org>,
        Dan Williams <dan.j.williams@...el.com>,
        Miguel Ojeda <ojeda@...nel.org>,
        Isabella Basso <isabbasso@...eup.net>,
        Vlastimil Babka <vbabka@...e.cz>,
        Rasmus Villemoes <linux@...musvillemoes.dk>,
        linux-kernel@...r.kernel.org, linux-kbuild@...r.kernel.org,
        llvm@...ts.linux.dev
Subject: Re: [PATCH v2] Makefile.debug: support for -gz=zstd

On Mon, Nov 7, 2022 at 7:08 PM Nick Desaulniers <ndesaulniers@...gle.com> wrote:
>
> Make DEBUG_INFO_COMPRESSED a choice; DEBUG_INFO_UNCOMPRESSED is the
> default, DEBUG_INFO_COMPRESSED uses zlib, DEBUG_INFO_COMPRESSED_ZSTD
> uses zstd.
>
> Some quick N=1 measurements with du, /usr/bin/time -v, and bloaty:
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_UNCOMPRESSED=y:
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:55.43
> 488M vmlinux
> 27.6%   136Mi   0.0%       0    .debug_info
>  6.1%  30.2Mi   0.0%       0    .debug_str_offsets
>  3.5%  17.2Mi   0.0%       0    .debug_line
>  3.3%  16.3Mi   0.0%       0    .debug_loclists
>  0.9%  4.62Mi   0.0%       0    .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED=y (zlib):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 1:00.35
> 385M vmlinux
> 21.8%  85.4Mi   0.0%       0    .debug_info
>  2.1%  8.26Mi   0.0%       0    .debug_str_offsets
>  2.1%  8.24Mi   0.0%       0    .debug_loclists
>  1.9%  7.48Mi   0.0%       0    .debug_line
>  0.5%  1.94Mi   0.0%       0    .debug_str
>
> clang-16, x86_64 defconfig plus
> CONFIG_DEBUG_INFO=y CONFIG_DEBUG_INFO_COMPRESSED_ZSTD=y (zstd):
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:59.69
> 373M vmlinux
> 21.4%  81.4Mi   0.0%       0    .debug_info
>  2.3%  8.85Mi   0.0%       0    .debug_loclists
>  1.5%  5.71Mi   0.0%       0    .debug_line
>  0.5%  1.95Mi   0.0%       0    .debug_str_offsets
>  0.4%  1.62Mi   0.0%       0    .debug_str
>
> That's only a 3.11% overall binary size savings over zlib, but at no
> performance regression.
>
> This renames the existing KConfig options:
> * DEBUG_INFO_UNCOMPRESSED -> DEBUG_INFO_COMPRESSED_NONE
> * DEBUG_INFO_COMPRESSED -> DEBUG_INFO_COMPRESSED_ZLIB
> So users upgrading may need to reset the new Kconfigs.
>
> Link: https://maskray.me/blog/2022-09-09-zstd-compressed-debug-sections
> Link: https://maskray.me/blog/2022-01-23-compressed-debug-sections
> Suggested-by: Sedat Dilek (DHL Supply Chain) <sedat.dilek@....com>
> Reviewed-by: Nathan Chancellor <nathan@...nel.org>
> Signed-off-by: Nick Desaulniers <ndesaulniers@...gle.com>
> ---
> Changes v1 -> v2:
> * Remove `depends on DEBUG_KERNEL` as per Nathan.
> * Rename Kconfigs as per Sedat and Masahiro.
> * Add note about renamed Kconfigs to commit message.
> * Add more help text to DEBUG_INFO_COMPRESSED_ZSTD.
>
>  lib/Kconfig.debug      | 29 +++++++++++++++++++++++++++--
>  scripts/Makefile.debug |  4 ++++
>  2 files changed, 31 insertions(+), 2 deletions(-)
>
> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
> index 29280072dc0e..7c28a8fba02e 100644
> --- a/lib/Kconfig.debug
> +++ b/lib/Kconfig.debug
> @@ -312,8 +312,21 @@ config DEBUG_INFO_REDUCED
>           DEBUG_INFO build and compile times are reduced too.
>           Only works with newer gcc versions.
>
> -config DEBUG_INFO_COMPRESSED
> -       bool "Compressed debugging information"
> +choice
> +       prompt "Compressed Debug information"
> +       help
> +         Compress the resulting debug info. Results in smaller debug info sections,
> +         but requires that consumers are able to decompress the results.
> +
> +         If unsure, choose DEBUG_INFO_COMPRESSED_NONE.
> +
> +config DEBUG_INFO_COMPRESSED_NONE
> +       bool "Don't compress debug information"
> +       help
> +         Don't compress debug info sections.
> +
> +config DEBUG_INFO_COMPRESSED_ZLIB
> +       bool "Compress debugging information with zlib"
>         depends on $(cc-option,-gz=zlib)
>         depends on $(ld-option,--compress-debug-sections=zlib)
>         help
> @@ -327,6 +340,18 @@ config DEBUG_INFO_COMPRESSED
>           preferable to setting $KDEB_COMPRESS to "none" which would be even
>           larger.
>
> +config DEBUG_INFO_COMPRESSED_ZSTD
> +       bool "Compress debugging information with zstd"
> +       depends on $(cc-option,-gz=zstd)
> +       depends on $(ld-option,--compress-debug-sections=zstd)
> +       help
> +         Compress the debug information using zstd.  This may provide better
> +         compression than zlib, for about the same time costs, but requires newer
> +         toolchain support.  Requires GCC 13.0+ or Clang 16.0+, binutils 2.40+, and
> +         zstd.
> +
> +endchoice # "Compressed Debug information"
> +
>  config DEBUG_INFO_SPLIT
>         bool "Produce split debuginfo in .dwo files"
>         depends on $(cc-option,-gsplit-dwarf)
> diff --git a/scripts/Makefile.debug b/scripts/Makefile.debug
> index 332c486f705f..8ac3379d2255 100644
> --- a/scripts/Makefile.debug
> +++ b/scripts/Makefile.debug
> @@ -31,6 +31,10 @@ ifdef CONFIG_DEBUG_INFO_COMPRESSED

^^ ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZLIB

...from a quick (re)view.

-Sedat-

>  DEBUG_CFLAGS   += -gz=zlib
>  KBUILD_AFLAGS  += -gz=zlib
>  KBUILD_LDFLAGS += --compress-debug-sections=zlib
> +else ifdef CONFIG_DEBUG_INFO_COMPRESSED_ZSTD
> +DEBUG_CFLAGS   += -gz=zstd
> +KBUILD_AFLAGS  += -gz=zstd
> +KBUILD_LDFLAGS += --compress-debug-sections=zstd
>  endif
>
>  KBUILD_CFLAGS  += $(DEBUG_CFLAGS)
>
> base-commit: f0c4d9fc9cc9462659728d168387191387e903cc
> --
> 2.38.1.431.g37b22c650d-goog
>

Powered by blists - more mailing lists