linux-kernel - Re: [PATCH V4 00/15] blk-throttle: add .high limit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <1bf8605a-9920-19ea-e2ab-2f3d747e55be@sandisk.com>
Date:   Mon, 14 Nov 2016 17:18:28 -0800
From:   Bart Van Assche <bart.vanassche@...disk.com>
To:     Shaohua Li <shli@...com>
CC:     "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Kernel-team@...com" <Kernel-team@...com>,
        "axboe@...com" <axboe@...com>, "tj@...nel.org" <tj@...nel.org>,
        "vgoyal@...hat.com" <vgoyal@...hat.com>
Subject: Re: [PATCH V4 00/15] blk-throttle: add .high limit

On 11/14/2016 04:49 PM, Shaohua Li wrote:
> On Mon, Nov 14, 2016 at 04:41:33PM -0800, Bart Van Assche wrote:
>> Thank you for pointing me to the discussion thread about v3 of this patch
>> series. Did I see correctly that one of the conclusions was that for users
>> this mechanism is hard to configure? Are we providing a good service to
>> Linux users by providing a mechanism that is hard to configure?
>
> Yes, this is a kind of low level knob and is expected to be configured by
> experienced users. This sucks, but we really don't have good solutions. If
> anybody has better ideas, I'm happy to try.

Hello Shaohua,

An approach I have been considering to analyze further is as follows:
* For rotational media use an algorithm like BFQ to preserve 
sequentiality of workloads and to guarantee fairness. This means that 
one application submits I/O per time slot.
* For SSDs, multiplex I/O from multiple applications during a single 
time slot to keep the queue depth high. Throttle I/O if needed to 
realize fairness.

Implementing this approach requires an approach for estimating I/O cost 
based on the request characteristics (offset and size) and the device 
type (rotational or SSD). This may require measuring the time that was 
needed to process past requests and to use that information in a 
learning algorithm.

Unless someone can convince me of the opposite I think that coming up 
with an algorithm for estimating I/O cost is essential to guarantee I/O 
fairness without requesting users to perform complicated parameter 
configurations.

Bart.