linux-kernel - Re: [RFC 0/3] block: proportional based blk-throttling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160122191144.GA2151859@devbig084.prn1.facebook.com>
Date:	Fri, 22 Jan 2016 11:11:45 -0800
From:	Shaohua Li <shli@...com>
To:	Tejun Heo <tj@...nel.org>
CC:	<linux-kernel@...r.kernel.org>, <axboe@...nel.dk>,
	<vgoyal@...hat.com>, <jmoyer@...hat.com>, <Kernel-team@...com>
Subject: Re: [RFC 0/3] block: proportional based blk-throttling

On Fri, Jan 22, 2016 at 01:08:44PM -0500, Tejun Heo wrote:
> Hello, Shaohua.
> 
> On Fri, Jan 22, 2016 at 09:57:10AM -0800, Shaohua Li wrote:
> > > Let's say per-cgroup buffer budget B is calculated as, say, 100ms
> > > worth of IO cost (or bandwidth or iops) available to the cgroup.  In
> > > practice, this may have to be adjusted down depending on the number of
> > > cgroups performing active IOs.  For a given cgroup, B can be
> > > distributed among the CPUs that are actively issuing IOs in that
> > > cgroup.  It will degenerate to round robin of small budget if there
> > > are too many active for the budget available but for most cases this
> > > will cut down most of cross-CPU traffic.
> > 
> > The cgroup could be a single thread. It uses cpu0's per-cpu budget B-1,
> > move to cpu1 and use another B - 1, and so on
> 
> Sure, just ensure that the total cached is bound by B and expire if
> not used over a certain amount of time.  The thing is as long as we
> can go through percpu cache most of the time, it's all fine.  We can
> spend a lot of processing budget for corner cases.
> 
> > >  cost = F + R * size
> > 
> > F could be IOPS. and the real cost becomes R. How do you get R? We can't
> > simply use R(4k) = 1, R(8k) = 2 .... I tried the idea several years ago:
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__lwn.net_Articles_474164_&d=CwIBAg&c=5VD0RTtNlTh3ycd41b3MUw&r=X13hAPkxmvBro1Ug8vcKHw&m=4X56EQmXhfF82BH-eQkQL08afWwbrOErtEVkn5xKsWA&s=_IkvDWMM7AXgh840OrQKndkJpBVcKrGhgLnHkA_aYNg&e= 
> > The idea is the same. But the reality is we can't get R. I don't want to
> > have a random math working for one SSD but not for another.
> 
> Yeah, it'll have to be adaptive.  We can't use fixed values; however,
> note that using bandwidth means that we assume F == 0 and R == 1,
> which wouldn't be appropriate for most devices.

It's true bandwidth means R == 1. But it has a kind of adaptive. The
cgroup bandwidth == share * disk_bandwidth. disk_bandwidth is adaptive.
It might not work well if cgroups have completely different IO pattern
though. 

Thanks,
Shaohua