linux-kernel - Re: [PATCH] erofs: add direct I/O support for compressed data

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <7f41c935-c352-45c5-8a24-d690755d67d3@linux.alibaba.com>
Date: Mon, 22 Sep 2025 22:43:29 +0800
From: Gao Xiang <hsiangkao@...ux.alibaba.com>
To: Chunhai Guo <guochunhai@...o.com>, xiang@...nel.org
Cc: chao@...nel.org, zbestahu@...il.com, jefflexu@...ux.alibaba.com,
 dhavale@...gle.com, lihongbo22@...wei.com, linux-erofs@...ts.ozlabs.org,
 linux-kernel@...r.kernel.org
Subject: Re: [PATCH] erofs: add direct I/O support for compressed data

Hi Chunhai,

On 2025/9/22 20:43, Chunhai Guo wrote:
> Direct I/O is particularly useful in memory-sensitive scenarios, as it
> provides more predictable performance by avoiding unnecessary page cache
> overheads. For example, when accessing large files such as AI model files
> that are typically read only once, buffered I/O introduces redundant page
> cache usage and extra page copies, leading to unstable performance and
> increased CPU load due to memory reclaim. While Direct I/O can avoid these.
> 
> The table below shows that the performance of direct I/O is up to 54.6%
> higher than buffered I/O in the low-memory scenario. The results were
> obtained using the fio benchmark with 8 threads, each thread read a 2.5GB
> file, on ARM64 Android devices running the 6.6 kernel with an 8-core CPU
> and 12GB of memory.
> 
> +--------------------------------------------------------------------------+
> | fio benchmark       | buffered I/O (MiB/s) | direct I/O (MiB/s) |  diff  |
> |---------------------+----------------------+--------------------+--------|
> | normal scenario     |        2629.8        |       3648.7       | +38.7% |
> |---------------------+----------------------+--------------------+--------|
> | low memory scenario |        2350.0        |       3633.9       | +54.6% |
> +--------------------------------------------------------------------------+

Thanks for your patch!

Yes, avoid page cache by using direct I/O for read-once data (e.g.
distributing huge LLM model) actually makes sense on my side and your test
result is impressive.

I will look into your implementation later, since it's too late for v6.18.
Let's address this feature for the v6.19 cycle.

Thanks,
Gao Xiang

> 
> This patch does not support the following two cases. They will fall back to
> buffered I/O:
> (1) large folios, which will be supported in a follow-up patch.
> (2) folios with private data attached, as the private data is required by
> this direct I/O implementation.
> 
> Signed-off-by: Chunhai Guo <guochunhai@...o.com>