linux-kernel - Re: [PATCH 00/10] erofs: 48-bit layout support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <751a34e5-5305-4c7f-9568-57cb6931c51b@linux.alibaba.com>
Date: Thu, 13 Mar 2025 20:32:09 +0800
From: Gao Xiang <hsiangkao@...ux.alibaba.com>
To: Chao Yu <chao@...nel.org>, linux-erofs@...ts.ozlabs.org
Cc: LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 00/10] erofs: 48-bit layout support



On 2025/3/13 20:06, Chao Yu wrote:
> On 2025/3/10 17:54, Gao Xiang wrote:
>> Hi folks,
>>
>> The current 32-bit block addressing limits EROFS to a 16TiB maximum
>> volume size with 4KiB blocks.  However, several new use cases now
>> require larger capacity support:
>>   - Massive datasets for model training to boost random sampling
>>     performance for each epoch;
>>   - Object storage clients using EROFS direct passthrough.
>>
>> This extends core on-disk structures to support 48-bit block addressing,
>> such as inodes, device slots, and inode chunks.
>>
>> In addition, it introduces an mtime field to 32-byte compact inodes for
>> basic timestamp support, as well as expands the superblock root NID to
>> an 8-byte rootnid_8b for out-of-place update incremental builds.
>>
>> In order to support larger images beyond 32-bit block addressing and
>> efficient indexing of large compression units for compressed data, and
>> to better support popular compression algorithms (mainly Zstd) that
>> still lack native fixed-sized output compression support, introduce
>> byte-oriented encoded extents, so that these compressors are allowed
>> to retain their current methods.
>>
>> Therefore, it speeds up Zstd image building a lot by using:
>> Processor: Intel(R) Xeon(R) Platinum 8163 CPU @ 2.50GHz * 96
>> Dataset: enwik9
>> Build time Size Type Command Line
>> 3m52.339s 266653696 FO -C524288 -zzstd,22
>> 3m48.549s 266174464 FO -E48bit -C524288 -zzstd,22
>> 0m12.821s 272134144 FI -E48bit -C1048576 --max-extent-bytes=1048576 -zzstd,22
>>
>> It has been stress-tested on my local setup for a while without any
>> strange happens.
> 
> Cool, good work! For this serial,
> 
> Acked-by: Chao Yu <chao@...nel.org>
> 
> Hoping to grab continuous free slots to check the details, then we can
> change it to rvb status before merge window. :)

Thanks, I don't find any issue so far and users would like to
get 48-bit addressing and quick Zstd compression. It's not a
complex update anyway.

Thanks,
Gao Xiang