linux-kernel - Re: [RFC PATCH v1 0/5] Introduce objects folding mechanism

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20230502103752.sb3mwkdqhdzxlt65@cab-wsm-0029881>
Date:   Tue, 2 May 2023 10:38:02 +0000
From:   Романов Алексей Васильевич 
        <AVRomanov@...rdevices.ru>
To:     "minchan@...nel.org" <minchan@...nel.org>,
        "senozhatsky@...omium.org" <senozhatsky@...omium.org>,
        "akpm@...ux-foundation.org" <akpm@...ux-foundation.org>
CC:     "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        kernel <kernel@...rdevices.ru>
Subject: Re: [RFC PATCH v1 0/5] Introduce objects folding mechanism

Hello!

On Tue, Apr 18, 2023 at 09:24:58AM +0300, Alexey Romanov wrote:
> Hello!
> 
> This RFC series adds feature which allows fold identical
> zsmalloc objects into a single one.
> 
> Based on ZRAM version:
> https://lore.kernel.org/lkml/Y3w8%2Fq%2FHoSbqamoD@google.com/t/
> 
> Let's imagine that 3 objects with the same content got into zsmalloc:
> 
> +----------------+   +----------------+   +----------------+
> |    handle 1    |   |    handle 2    |   |    handle 3    |
> +-------+--------+   +-------+--------+   +--------+-------+
>         |                    |                     |
>         |                    |                     |
> +-------v--------+   +-------v---------+  +--------v-------+
> |zsmalloc  object|   |zsmalloc  object |  |zsmalloc  object|
> ++--------------++   +-+-------------+-+  ++--------------++
>  +--------------+      +-------------+     +--------------+
>  | buffer: "abc"|      |buffer: "abc"|     | buffer: "abc"|
>  +--------------+      +-------------+     +--------------+
> 
> As you can see, the data is duplicated. Fold mechanism saves
> (after scanning objects) only one zsmalloc object. Here's
> what happens after the scan and fold:
> 
> +----------------+   +----------------+   +----------------+
> |    handle 1    |   |    handle 2    |   |    handle 3    |
> +-------+--------+   +-------+--------+   +--------+-------+
>         |                    |                     |
>         |                    |                     |
>         |           +--------v---------+           |
>         +-----------> zsmalloc  object <-----------+
>                     +--+-------------+-+
>                        +-------------+
>                        |buffer: "abc"|
>                        +-------------+
> 
> Thus, we reduced the amount of memory occupied by 3 times.
> 
> This mechanism doesn't affect the perf of the zsmalloc itself in
> any way (maybe just a little bit on the zs_free() function).
> In order to describe each such identical object, we (constantly)
> need sizeof(fold_rbtree_node) bytes. Also, all struct size_class now
> have new field struct rb_root fold_rbtree.
> 
> Testing on my system (8GB RAM + 1Gb ZRAM SWAP) showed that at high
> loads, on average, when calling the fold mechanism, we can save
> up to 15-20% of the memory usage.
> 
> This patch series adds a new sysfs node into ZRAM - trigger folding
> and provides new field in mm_stat. This field shows how many pages
> freed during folding:
> 
>   $ cat /sys/block/zram0/mm_stat
>     431452160 332984392 339894272 0 339894272 282 0 51374 51374 0
> 
>   $ echo 1 > /sys/block/zram0/fold
> 
>   $ cat /sys/block/zram/mm_stat
>     431452160 270376848 287301504 0 339894272 282 0 51374 51374 6593
> 
> Alexey Romanov (5):
>   mm/zsmalloc: use ARRAY_SIZE in isolate_zspage()
>   mm/zsmalloc: get rid of PAGE_MASK
>   mm/zsmalloc: introduce objects folding mechanism
>   zram: add fold sysfs knob
>   zram: add pages_folded to stats
> 
>  Documentation/admin-guide/blockdev/zram.rst |   2 +
>  drivers/block/zram/zram_drv.c               |  30 +-
>  include/linux/zsmalloc.h                    |   4 +
>  mm/Kconfig                                  |   9 +
>  mm/zsmalloc.c                               | 484 +++++++++++++++++++-
>  5 files changed, 513 insertions(+), 16 deletions(-)
> 
> -- 
> 2.38.1
> 

Really sorry for the noise, but could you comment on my patchset and
results? We have moved away from the terms and concepts used in the
patent we discussed in the ZRAM version, and I believe we can safely use
these changes in zsmalloc.

-- 
Thank you,
Alexey