linux-kernel - Re: [PATCH v3 0/9] zram: introduce crypto decompress noctx API and use it on zram

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150921035812.GE27729@bbox>
Date:	Mon, 21 Sep 2015 12:58:12 +0900
From:	Minchan Kim <minchan@...nel.org>
To:	Joonsoo Kim <js1304@...il.com>
Cc:	Andrew Morton <akpm@...ux-foundation.org>,
	Nitin Gupta <ngupta@...are.org>,
	Sergey Senozhatsky <sergey.senozhatsky.work@...il.com>,
	linux-kernel@...r.kernel.org, linux-crypto@...r.kernel.org,
	Herbert Xu <herbert@...dor.apana.org.au>,
	"David S. Miller" <davem@...emloft.net>,
	Stephan Mueller <smueller@...onox.de>,
	Joonsoo Kim <iamjoonsoo.kim@....com>
Subject: Re: [PATCH v3 0/9] zram: introduce crypto decompress noctx API and
 use it on zram

On Fri, Sep 18, 2015 at 02:19:15PM +0900, Joonsoo Kim wrote:
> This patchset makes zram to use crypto API in order to support
> more compression algorithm.
> 
> The reason we need to support vairous compression algorithms is that
> zram's performance is highly depend on workload and compression algorithm
> and architecture. Every compression algorithm has it's own strong point.
> For example, zlib is the best for compression ratio, but, it's
> (de)compression speed is rather slow. Against my expectation, in my kernel
> build test with zram swap, in low-memory condition on x86, zlib shows best
> performance than others. In this case, I guess that compression ratio is
> the most important factor. Unlike this situation, on ARM, maybe fast
> (de)compression speed is the most important because it's computation speed
> is slower than x86.

Fair enough. lzo and lz4 cannot cover all of workloads so surely, there are
workloads other compressor algorithm can win. As well, we could support
H/W compressor with crypto support so I think crypto is a way to go.

One thing I have a concern is I don't want to bind some of compressor
algorithm to zram statically. User can see what kinds of crypto compressor
support in his system via /proc/crypto and use it dynamically.
I hope zram supports it.

> 
> Anyway, there is a concern from Sergey to use crypto API in zram. Current
> crypto API has a limitation that always require tfm object to (de)compress
> something because some of (de)compression function requires scratch buffer
> embedded on tfm even if some of (de)compression function doesn't
> require it. Due to above reason, using crypto API rather than calling

Nice catch from Sergey!

> compression library directly causes more memory footprint and this is
> why zram doesn't use crypto API before.
> 
> In this patchset, crypto compress noctx API is introduced to reduce memory
> footprint caused by maintaining multiple tfm and zram uses it. Before
> noctx API, zram's performace is down-graded if tfm is insufficient. But,
> after applying noctx API, performace is restored.
> 
> This addresses Sergey's concern perfectly and provides possibility to use
> various compression algorithm in zram.
> 
> Following is zram's read performance number.
> 
> * iozone -t 4 -R -r 16K -s 60M -I +Z -i 0 -i 1
> * max_stream is set to 1
> * Output is in Kbytes/sec
> 
> zram-base vs zram-crypto vs zram-crypto-noctx
> 
> Read		10411701.88	6426911.62	9423894.38 
> Re-read		10017386.62	6428218.88	11000063.50 

Another patch and number we need to merge this patchset is zlib support
patchset and number you had experiement.

> 
> Thanks.
> 
> Joonsoo Kim (7):
>   crypto: introduce decompression API that can be called via sharable
>     tfm object
>   crypto/lzo: support decompress_noctx
>   crypyo/lz4: support decompress_noctx
>   crypto/lz4hc: support decompress_noctx
>   crypto/842: support decompress_noctx
>   zram: use crypto API for compression
>   zram: use crypto decompress_noctx API
> 
> Sergey Senozhatsky (2):
>   zram: make stream find and release functions static
>   zram: pass zstrm down to decompression path
> 
>  crypto/842.c                   |   9 +++-
>  crypto/compress.c              |  36 +++++++++++++++
>  crypto/crypto_null.c           |   3 +-
>  crypto/deflate.c               |   3 +-
>  crypto/lz4.c                   |   9 +++-
>  crypto/lz4hc.c                 |   9 +++-
>  crypto/lzo.c                   |   9 +++-
>  drivers/block/zram/Kconfig     |   8 ++--
>  drivers/block/zram/Makefile    |   4 +-
>  drivers/block/zram/zcomp.c     | 102 +++++++++++++++++++++++++++++++----------
>  drivers/block/zram/zcomp.h     |  42 +++++++----------
>  drivers/block/zram/zcomp_lz4.c |  47 -------------------
>  drivers/block/zram/zcomp_lz4.h |  17 -------
>  drivers/block/zram/zcomp_lzo.c |  47 -------------------
>  drivers/block/zram/zcomp_lzo.h |  17 -------
>  drivers/block/zram/zram_drv.c  |  32 +++++++++----
>  include/linux/crypto.h         |  20 ++++++++
>  17 files changed, 211 insertions(+), 203 deletions(-)
>  delete mode 100644 drivers/block/zram/zcomp_lz4.c
>  delete mode 100644 drivers/block/zram/zcomp_lz4.h
>  delete mode 100644 drivers/block/zram/zcomp_lzo.c
>  delete mode 100644 drivers/block/zram/zcomp_lzo.h
> 
> -- 
> 1.9.1
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/