linux-kernel - Re: [PATCH v9 1/4] blk-throttle: fix that io throttle can only work for single bio

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <Yww6lPQaz/Lk4lL6@slm.duckdns.org>
Date:   Sun, 28 Aug 2022 18:03:32 -1000
From:   Tejun Heo <tj@...nel.org>
To:     Yu Kuai <yukuai1@...weicloud.com>
Cc:     axboe@...nel.dk, mkoutny@...e.com, ming.lei@...hat.com,
        linux-block@...r.kernel.org, linux-kernel@...r.kernel.org,
        cgroups@...r.kernel.org, yukuai3@...wei.com, yi.zhang@...wei.com
Subject: Re: [PATCH v9 1/4] blk-throttle: fix that io throttle can only work
 for single bio

On Mon, Aug 29, 2022 at 10:22:37AM +0800, Yu Kuai wrote:
> From: Yu Kuai <yukuai3@...wei.com>
> 
> Test scripts:
> cd /sys/fs/cgroup/blkio/
> echo "8:0 1024" > blkio.throttle.write_bps_device
> echo $$ > cgroup.procs
> dd if=/dev/zero of=/dev/sda bs=10k count=1 oflag=direct &
> dd if=/dev/zero of=/dev/sda bs=10k count=1 oflag=direct &
> 
> Test result:
> 10240 bytes (10 kB, 10 KiB) copied, 10.0134 s, 1.0 kB/s
> 10240 bytes (10 kB, 10 KiB) copied, 10.0135 s, 1.0 kB/s
> 
> The problem is that the second bio is finished after 10s instead of 20s.
> 
> Root cause:
> 1) second bio will be flagged:
> 
> __blk_throtl_bio
>  while (true) {
>   ...
>   if (sq->nr_queued[rw]) -> some bio is throttled already
>    break
>  };
>  bio_set_flag(bio, BIO_THROTTLED); -> flag the bio
> 
> 2) flagged bio will be dispatched without waiting:
> 
> throtl_dispatch_tg
>  tg_may_dispatch
>   tg_with_in_bps_limit
>    if (bps_limit == U64_MAX || bio_flagged(bio, BIO_THROTTLED))
>     *wait = 0; -> wait time is zero
>     return true;
> 
> commit 9f5ede3c01f9 ("block: throttle split bio in case of iops limit")
> support to count split bios for iops limit, thus it adds flagged bio
> checking in tg_with_in_bps_limit() so that split bios will only count
> once for bps limit, however, it introduce a new problem that io throttle
> won't work if multiple bios are throttled.
> 
> In order to fix the problem, handle iops/bps limit in different ways:
> 
> 1) for iops limit, there is no flag to record if the bio is throttled,
>    and iops is always applied.
> 2) for bps limit, original bio will be flagged with BIO_BPS_THROTTLED,
>    and io throttle will ignore bio with the flag.
> 
> Noted this patch also remove the code to set flag in __bio_clone(), it's
> introduced in commit 111be8839817 ("block-throttle: avoid double
> charge"), and author thinks split bio can be resubmited and throttled
> again, which is wrong because split bio will continue to dispatch from
> caller.
> 
> Fixes: 9f5ede3c01f9 ("block: throttle split bio in case of iops limit")
> Cc: <stable@...r.kernel.org>
> Signed-off-by: Yu Kuai <yukuai3@...wei.com>

Acked-by: Tejun Heo <tj@...nel.org>

Thanks.

-- 
tejun