linux-kernel - Re: [PATCH 00/10] erofs: 48-bit layout support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <695b5311-d0bb-422c-9a96-52cfcc72afb4@kernel.org>
Date: Thu, 13 Mar 2025 20:06:58 +0800
From: Chao Yu <chao@...nel.org>
To: Gao Xiang <hsiangkao@...ux.alibaba.com>, linux-erofs@...ts.ozlabs.org
Cc: chao@...nel.org, LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 00/10] erofs: 48-bit layout support

On 2025/3/10 17:54, Gao Xiang wrote:
> Hi folks,
> 
> The current 32-bit block addressing limits EROFS to a 16TiB maximum
> volume size with 4KiB blocks.  However, several new use cases now
> require larger capacity support:
>   - Massive datasets for model training to boost random sampling
>     performance for each epoch;
>   - Object storage clients using EROFS direct passthrough.
> 
> This extends core on-disk structures to support 48-bit block addressing,
> such as inodes, device slots, and inode chunks.
> 
> In addition, it introduces an mtime field to 32-byte compact inodes for
> basic timestamp support, as well as expands the superblock root NID to
> an 8-byte rootnid_8b for out-of-place update incremental builds.
> 
> In order to support larger images beyond 32-bit block addressing and
> efficient indexing of large compression units for compressed data, and
> to better support popular compression algorithms (mainly Zstd) that
> still lack native fixed-sized output compression support, introduce
> byte-oriented encoded extents, so that these compressors are allowed
> to retain their current methods.
> 
> Therefore, it speeds up Zstd image building a lot by using:
> Processor: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz * 96
> Dataset: enwik9
> Build time Size Type Command Line
> 3m52.339s 266653696 FO -C524288 -zzstd,22
> 3m48.549s 266174464 FO -E48bit -C524288 -zzstd,22
> 0m12.821s 272134144 FI -E48bit -C1048576 --max-extent-bytes=1048576 -zzstd,22
> 
> It has been stress-tested on my local setup for a while without any
> strange happens.

Cool, good work! For this serial,

Acked-by: Chao Yu <chao@...nel.org>

Hoping to grab continuous free slots to check the details, then we can
change it to rvb status before merge window. :)

Thanks,

> 
> Thanks,
> Gao Xiang
> 
> Gao Xiang (10):
>    erofs: get rid of erofs_map_blocks_flatmode()
>    erofs: simplify erofs_{read,fill}_inode()
>    erofs: add 48-bit block addressing on-disk support
>    erofs: implement 48-bit block addressing for unencoded inodes
>    erofs: support dot-omitted directories
>    erofs: initialize decompression early
>    erofs: add encoded extent on-disk definition
>    erofs: implement encoded extent metadata
>    erofs: support unaligned encoded data
>    erofs: enable 48-bit layout support
> 
>   fs/erofs/Kconfig             |  14 +--
>   fs/erofs/data.c              | 133 +++++++++++-------------
>   fs/erofs/decompressor.c      |   2 +-
>   fs/erofs/dir.c               |   7 +-
>   fs/erofs/erofs_fs.h          | 191 ++++++++++++++++-------------------
>   fs/erofs/inode.c             | 126 +++++++++++------------
>   fs/erofs/internal.h          |  30 +++---
>   fs/erofs/super.c             |  49 ++++-----
>   fs/erofs/sysfs.c             |   2 +
>   fs/erofs/zdata.c             |  96 +++++++++---------
>   fs/erofs/zmap.c              | 166 +++++++++++++++++++++++++-----
>   include/trace/events/erofs.h |   2 +-
>   12 files changed, 455 insertions(+), 363 deletions(-)
>