Date:	Wed, 14 May 2014 02:35:42 +0100
From:	Phillip Lougher <phillip@...gher.demon.co.uk>
To:	linux-fsdevel@...r.kernel.org
CC:	Linux Kernel Development <linux-kernel@...r.kernel.org>
Subject: [ANN] Squashfs tools 4.3 released

Hi,

I'm pleased to announce the release of Squashfs tools 4.3.
This is the first release in over 3 years, and there are substantial
improvements to stability, new compression options and compressors,
speed optimisations, new options for Mksquashfs/Unsquashfs, and fixes
for CVE-2012-4024 and CVE-2012-4025.

The new release can be downloaded here:
http://sourceforge.net/projects/squashfs/files/latest/download?source=files

Changes in Squashfs-tools 4.3
-----------------------------

1. Stability improvements.  Better checking of user input for out of
    range/invalid values.  Better handling of corrupted Squashfs filesystems
    (Mksquashfs append mode, and Unsquashfs).  Better handling of buffer
    overflow/underflow.

2. GZIP compressor now supports compression options, allowing different
    compression levels to be used.

3. Rewritten LZO compressor with compression options, allowing different
    LZO algorithms and different compression levels to be used.

4. New LZ4 compressor (note not yet in mainline kernel)

5. Better default memory usage for Mksquashfs.  Mksquashfs by default now
    uses 25% of physical memory.

6. Duplicate checking in Mksquashfs further optimised.  With certain
    "problem filesystems" greater than 2x performance improvement.
    Filesystems with a lot of duplicates should see at least 10-20% speed
    improvement.

7. The -stat option in Unsquashfs now displays the compression options
    used to generate the original filesystem.  Previously -stat only displayed
    the compression algorithm used.

8. The file being compressed/uncompressed in Mksquashfs/Unsquashfs is now
    displayed if CTRL-\ (SIGQUIT from keyboard) typed.

9. The status of the internal queues/caches in Mksquashfs/Unsquashfs is
    now displayed if CTRL-\ (SIGQUIT from keyboard) is typed twice within
    one second.  Normally only useful for "power users", but it can be
    used to discover if there's any bottlenecks affecting performance
    (the bottleneck will normally be the compressors/fragment compressors).

10. Miscellaneous new options for Mksquashfs/Unsquashfs to fine tune behaviour.

11. Fixes for CVE-2012-4024 and CVE-2012-4025.

Compatiblity
------------

Mksquashfs 4.3 generates 4.0 filesystems.  These filesystems are fully
compatible/interchangable with filesystems generated by Mksquashfs 4.0 and are
mountable on 2.6.29 and later kernels.

Compressors
-----------

New compression options and compressors are now supported.

The new options and compressors are:

1. gzip
	  -Xcompression-level <compression-level>
		<compression-level> should be 1 .. 9 (default 9)
	  -Xwindow-size <window-size>
		<window-size> should be 8 .. 15 (default 15)
	  -Xstrategy strategy1,strategy2,...,strategyN
		Compress using strategy1,strategy2,...,strategyN in turn
		and choose the best compression.
		Available strategies: default, filtered, huffman_only,
		run_length_encoded and fixed

2. lzo
	  -Xalgorithm <algorithm>
		Where <algorithm> is one of:
			lzo1x_1
			lzo1x_1_11
			lzo1x_1_12
			lzo1x_1_15
			lzo1x_999 (default)
	  -Xcompression-level <compression-level>
		<compression-level> should be 1 .. 9 (default 8)
		Only applies to lzo1x_999 algorithm

3. lz4
	  -Xhc
		Compress using LZ4 High Compression

The compression specific options are, obviously, specific to the compressor
in question, and you should read the compressor documentation and check
their web sites to understand their behaviour.

In general the defaults used by Mksquashfs for each compressor are optimised
to give the best performance for each compressor, where what constitutes
best depends on the compressor.  For gzip/xz best means highest compression
(trying multiple filters/strategies can improve compression, but this is
extremely expensive computationally, and hence, not suitable for the defaults),
for LZO/LZ4 best means a tradeoff between compression and (de)-compression
overhead (LZO/LZ4 by definition are intended for weaker processors).

New Mksquashfs options
----------------------

1. -mem <size>

   Set the amount of memory used by Mksquashfs to <size> bytes.  G/M and K
   post-fixes are supported.

   By default Mksquashfs uses 25% of the physical memory.  Increasing
   this with the -mem option can increase performance (note it does not have
   any effect on compression).  Reducing it can prevent thrashing if the
   system is busy and there is not 25% of physical memory free (again, note
   it does not have any effect on compression).

2. -exit-on-error

   By default Mksquashfs treats certain errors as benign, if these
   errors occur Mksquashfs prints the error on the console but continues.
   These errors are typically failure to read a file from the source filesystem.
   This is deliberate, in many cases users prefer Mksquashfs to flag
   the error but continue rather than abort what may be hours of compression.

   But there are times where failure to read any file is considered critical,
   and users (especially in the case of automated scripts where the
   errors output to the console may be missed) prefer Mksquashfs to exit.

   The new -exit-on-error option can be used in this scenario.  This option
   makes Mksquashfs treat all benign errors as fatal.

3. -progress

   By default if -info is specified, the progress bar is disabled as it gets
   in the way.  Occasionally you might want the progress bar enabled whilst
   -info is enabled.  This option forces Mksquashfs to output the progress
   bar when -info is specified.

4. -Xhelp

   Display the usage text for the currently selected compressor.

New Unsquashfs options
----------------------

1. -u[ser-xattrs]

   Only write user xattrs.  This forces Unsquashfs to ignore system xattrs.
   This is useful when Unsquashing a filesystem as a non-root user, and the
   filesystem contains system xattrs which are only writable by root.

Major bugs fixed
----------------

1. If Mksquashfs ran out of space in the destination filesystem, this
    would not cause Mksquashfs to immediately abort, and Mksquashfs would
    continue to process the source filesystem.  Mksquashfs now immediately
    aborts on out of space in the destination filesystem.

2. Unsquashfs ignored the maximum number of open files limit, and if that
    was lower than the default limit for Linux, it would run out of file
    descriptors.  Unsquashfs now limits the number of open files to the
    limit currently in force (e.g. specified by setrlimit).

3. If huge numbers of dynamic pseudo files were specified, Mksquashfs
    could exceed the maximum number of open files limit.  This was because
    Mksquashfs created all the dynamic file processes up front before
    commencing source filesystem reading and compression.  Mksquashfs
    now creates the dynamic file processes on demand whilst reading
    and compressing the source filesystem, thus limiting the number of
    dynamic pseudo file processes in existence at any one time.

4. When outputting Unsquashfs used to set the permissions of directories
    as it recursively descended.  This in hindsight had an obvious oversight,
    if a directory had only read permission (or was otherwise restricted), then
    Unsquashfs would fail to write its contents when descending into it.  Fixed
    by setting directory permissions as Unsquashfs recursively unwinds.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists