linux-kernel - [PATCH 1/2] fs/squashfs: Make SquashFS xz initialization mode configurable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Date:   Fri,  8 Nov 2019 12:00:39 -0800
From:   Daniel Walker <danielwa@...co.com>
To:     Phillip Lougher <phillip@...ashfs.org.uk>
Cc:     "yusun2@...co.com" <yusun2@...co.com>, xe-linux-external@...co.com,
        Daniel Walker <dwalker@...o99.com>,
        linux-kernel@...r.kernel.org
Subject: [PATCH 1/2] fs/squashfs: Make SquashFS xz initialization mode configurable

From: "yusun2@...co.com" <yusun2@...co.com>

Make SquashFS xz initialization mode configurable to be either
XZ_PREALLOC or XZ_DYNALLOC. The default mode is XZ_PREALLOC.

SquashFS multi-threaded per-CPU decompressor is proven to
effectively resolve the I/O bottleneck and boost the outcome
of other boot time optimization technologies on some I/O bound
platforms. However it allocates extra memory on per-processor
per-mounted-package basis. Making XZ_DYNALLOC mode an option
for the xz decompressor initialization in SquashFS minimizes
the memory impact.

Signed-off-by: Yu Sun <yusun2@...co.com>
Cc: xe-linux-external@...co.com
Signed-off-by: Daniel Walker <dwalker@...o99.com>
---
 fs/squashfs/Kconfig      | 32 ++++++++++++++++++++++++++++++++
 fs/squashfs/xz_wrapper.c |  6 +++++-
 2 files changed, 37 insertions(+), 1 deletion(-)

diff --git a/fs/squashfs/Kconfig b/fs/squashfs/Kconfig
index 916e78fabcaa..9cf2ebf89374 100644
--- a/fs/squashfs/Kconfig
+++ b/fs/squashfs/Kconfig
@@ -223,3 +223,35 @@ config SQUASHFS_FRAGMENT_CACHE_SIZE
 
 	  Note there must be at least one cached fragment.  Anything
 	  much more than three will probably not make much difference.
+
+choice
+    prompt "XZ decompressor operation mode"
+    depends on SQUASHFS_XZ
+    default SQUASHFS_XZ_DICT_PREALLOC
+    help
+      Squashfs now utilizes the two different multi-call modes of xz
+      decompression. They each exhibits various trade-offs between
+      decompression performance and memory consumption.
+
+      If in doubt, select "XZ preallocated multi-call mode"
+
+config SQUASHFS_XZ_DICT_PREALLOC
+    bool "XZ preallocated multi-call mode"
+    help
+      Traditionally Squashfs has used XZ_PREALLOC operation mode for
+      xz decompression, under which the xz dictionary buffer is allocated
+      at initialization.
+
+config SQUASHFS_XZ_DICT_DYNALLOC
+    bool "XZ dynamic-allocated multi-call mode"
+    help
+      By default Squashfs uses XZ_PREALLOC operation mode for xz decompressor.
+      This, however, potentially lead to significant increase of memory
+      consumption, especially when SquashFS per-cpu multi-threaded
+      decompressor is applied.
+
+      If the system has memory constraints, setting this option will force
+      SquashFS to use XZ_DYNALLOC mode, and thus reduce memory footprint.
+      In this case, the LZMA2 dictionary is allocated upon needed with the
+      required size.
+endchoice
diff --git a/fs/squashfs/xz_wrapper.c b/fs/squashfs/xz_wrapper.c
index 4b2f2051a6dc..6d6946bb5e4c 100644
--- a/fs/squashfs/xz_wrapper.c
+++ b/fs/squashfs/xz_wrapper.c
@@ -90,7 +90,11 @@ static void *squashfs_xz_init(struct squashfs_sb_info *msblk, void *buff)
 		goto failed;
 	}
 
-	stream->state = xz_dec_init(XZ_PREALLOC, comp_opts->dict_size);
+	if (IS_ENABLED(CONFIG_SQUASHFS_XZ_DICT_DYNALLOC)) {
+	    stream->state = xz_dec_init(XZ_DYNALLOC, comp_opts->dict_size);
+	} else {
+	    stream->state = xz_dec_init(XZ_PREALLOC, comp_opts->dict_size);
+	}
 	if (stream->state == NULL) {
 		kfree(stream);
 		err = -ENOMEM;
-- 
2.17.1