linux-kernel - Re: [RFC PATCH] blk-throtl: Introduce sync queue for write ios

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Y5et48VryiKgL/eD@slm.duckdns.org>
Date:   Mon, 12 Dec 2022 12:40:35 -1000
From:   Tejun Heo <tj@...nel.org>
To:     Jinke Han <hanjinke.666@...edance.com>
Cc:     josef@...icpanda.com, axboe@...nel.dk, cgroups@...r.kernel.org,
        linux-block@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [RFC PATCH] blk-throtl: Introduce sync queue for write ios

On Wed, Dec 07, 2022 at 12:38:26AM +0800, Jinke Han wrote:
> From: Jinke Han <hanjinke.666@...edance.com>
> 
> Now we don't distinguish sync write ios from normal buffer write ios
> in blk-throtl. A bio with REQ_SYNC tagged always mean it will be wait
> until write completion soon after it submit. So it's reasonable for sync
> io to complete as soon as possible.
> 
> In our test, fio writes a 100g file in sequential 4k blocksize in
> a container with low bps limit configured (wbps=10M). More than 1200
> ios were throttled in blk-throtl queue and the avarage throtle time
> of each io is 140s. At the same time, the operation of saving a small
> file by vim will be blocked amolst 140s. As a fsync will be send by vim,
> the sync ios of fsync will be blocked by a huge amount of buffer write
> ios ahead. This is also a priority inversion problem within one cgroup.
> In the database scene, things got really bad with blk-throtle enabled
> as fsync is called very often.
> 
> This patch introduces a independent sync queue for write ios and gives
> a huge priority to sync write ios. I think it's a nice respond to the
> semantics of REQ_SYNC. Bios with REQ_META and REQ_PRIO gains the same
> priority as they are important to fs. This may avoid some potential
> priority inversion problems.

I think the idea makes sense but wonder whether the implementation would be
cleaner / simpler if the sq->queued[] are indexed by SYNC, ASYNC and the
sync writes are queued in the sync queue together with reads.

Thanks.

-- 
tejun