linux-kernel - Re: [PATCH V4 00/15] blk-throttle: add .high limit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20161115012857.GA17239@shli-mbp.local>
Date:   Mon, 14 Nov 2016 17:28:58 -0800
From:   Shaohua Li <shli@...com>
To:     Bart Van Assche <bart.vanassche@...disk.com>
CC:     "linux-block@...r.kernel.org" <linux-block@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "Kernel-team@...com" <Kernel-team@...com>,
        "axboe@...com" <axboe@...com>, "tj@...nel.org" <tj@...nel.org>,
        "vgoyal@...hat.com" <vgoyal@...hat.com>
Subject: Re: [PATCH V4 00/15] blk-throttle: add .high limit

On Mon, Nov 14, 2016 at 05:18:28PM -0800, Bart Van Assche wrote:
> On 11/14/2016 04:49 PM, Shaohua Li wrote:
> > On Mon, Nov 14, 2016 at 04:41:33PM -0800, Bart Van Assche wrote:
> > > Thank you for pointing me to the discussion thread about v3 of this patch
> > > series. Did I see correctly that one of the conclusions was that for users
> > > this mechanism is hard to configure? Are we providing a good service to
> > > Linux users by providing a mechanism that is hard to configure?
> > 
> > Yes, this is a kind of low level knob and is expected to be configured by
> > experienced users. This sucks, but we really don't have good solutions. If
> > anybody has better ideas, I'm happy to try.
> 
> Hello Shaohua,
> 
> An approach I have been considering to analyze further is as follows:
> * For rotational media use an algorithm like BFQ to preserve sequentiality
> of workloads and to guarantee fairness. This means that one application
> submits I/O per time slot.
> * For SSDs, multiplex I/O from multiple applications during a single time
> slot to keep the queue depth high. Throttle I/O if needed to realize
> fairness.
> 
> Implementing this approach requires an approach for estimating I/O cost
> based on the request characteristics (offset and size) and the device type
> (rotational or SSD). This may require measuring the time that was needed to
> process past requests and to use that information in a learning algorithm.
> 
> Unless someone can convince me of the opposite I think that coming up with
> an algorithm for estimating I/O cost is essential to guarantee I/O fairness
> without requesting users to perform complicated parameter configurations.

That's what I tried before:
http://marc.info/?l=linux-kernel&m=145617863208940&w=2

Unfortunately estimating I/O cost and disk capability is very hard if not
impossible. People objected using bandwidth or iops to estimate I/O cost.

Thanks,
Shaohua