linux-ext4 - Re: lastest kernel ext4 io slowly

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20190220103826.GC27474@quack2.suse.cz>
Date:   Wed, 20 Feb 2019 11:38:26 +0100
From:   Jan Kara <jack@...e.cz>
To:     fanchaoting@...jie.com.cn
Cc:     jack@...e.cz, tytso@....edu, linux-ext4@...r.kernel.org
Subject: Re: lastest kernel ext4 io slowly

On Wed 20-02-19 06:23:09, fanchaoting@...jie.com.cn wrote:
> ############################patch#################################
> 
> commit 16c54688592ce8eea85d2a26d37b64fa07e3e233
> Author: Jan Kara <jack@...e.cz>
> Date:   Fri Sep 30 01:03:17 2016 -0400
> 
>     ext4: Allow parallel DIO reads
> 
>     We can easily support parallel direct IO reads. We only have to make
>     sure we cannot expose uninitialized data by reading allocated block to
>     which data was not written yet, or which was already truncated. That is
>     easily achieved by holding inode_lock in shared mode - that excludes all
>     writes, truncates, hole punches. We also have to guard against page
>     writeback allocating blocks for delay-allocated pages - that race is
>     handled by the fact that we writeback all the pages in the affected
>     range and the lock protects us from new pages being created there.
> 
>     Signed-off-by: Jan Kara <jack@...e.cz>
>     Signed-off-by: Theodore Ts'o <tytso@....edu>
> 
> ############################################################
> 
> 
> hi jack , when I checkout lastest kernel
> "https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.101.tar.xz".  I
> use fio to test it , I find io become slowly. Here is my test method:
> 
> # fio -filename=/fio_test -direct=1 -iodepth 1 -thread -rw=randrw
> -rwmixread=70 -ioengine=psync -bs=4k -size=5G -numjobs=50 -runtime=180
> -group_reporting -name=randrw_70read_4k
> 
...

> Run status group 0 (all jobs):
>    READ: bw=29.9MiB/s (31.3MB/s), 29.9MiB/s-29.9MiB/s (31.3MB/s-31.3MB/s), io=5377MiB (5638MB), run=180006-180006msec
>   WRITE: bw=12.8MiB/s (13.4MB/s), 12.8MiB/s-12.8MiB/s (13.4MB/s-13.4MB/s), io=2303MiB (2415MB), run=180006-180006msec
> 
> when I revert this patch, I find io is faster
...
> Run status group 0 (all jobs):
>    READ: bw=85.3MiB/s (89.4MB/s), 85.3MiB/s-85.3MiB/s (89.4MB/s-89.4MB/s), io=14.0GiB (16.1GB), run=180003-180003msec
>   WRITE: bw=36.6MiB/s (38.4MB/s), 36.6MiB/s-36.6MiB/s (38.4MB/s-38.4MB/s), io=6587MiB (6907MB), run=180003-180003msec

Yes, this is understandable because you direct all 50 threads to do mixed
read-write synchronous IO against the same file (I suppose you mount ext4
with dioread_nolock mount option, right?). This commit changes direct IO
code to use shared lock on the inode so now reads cannot run in parallel to
writes on this file.

Honestly, I find the testcase a bit artificial (applications like databases
that care about good throughput for mixed read-write workload use AIO which
avoids this issue) and the code is simpler this way. Do you have some real
use case for this?

								Honza
-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR