linux-kernel - Re: [RFC/RFT PATCH] cfq-iosched: Implement cfq group idling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 8 Jul 2010 09:56:25 -0400
From:	Vivek Goyal <vgoyal@...hat.com>
To:	Jeff Moyer <jmoyer@...hat.com>
Cc:	linux kernel mailing list <linux-kernel@...r.kernel.org>,
	Jens Axboe <axboe@...nel.dk>,
	Corrado Zoccolo <czoccolo@...il.com>,
	Nauman Rafique <nauman@...gle.com>,
	Divyesh Shah <dpshah@...gle.com>,
	Gui Jianfeng <guijianfeng@...fujitsu.com>
Subject: Re: [RFC/RFT PATCH] cfq-iosched: Implement cfq group idling

On Thu, Jul 08, 2010 at 09:39:45AM -0400, Jeff Moyer wrote:
> Vivek Goyal <vgoyal@...hat.com> writes:
> 
> > Currently we idle on sequential queues and allow dispatch from a single
> > queue and that can become a bottleneck on higher end storage. For example
> > on my HP EVA, I can run multiple sequential streams and achieve top BW
> > of around 350 MB/s. But with CFQ, dispatching from single queue does not
> > keep the array busy (limits to 150-180 MB/s with 4 or 8 processes).
> >
> > One approach to solve this issue is simply use slice_idle = 0. But this
> > also takes away the any service differentiation between groups.
> 
> That also takes away service differentiation between queues.  If you
> want to maintain that at all, then this is really just pushing the
> problem to another layer.
> 

Yes it does take away the io priority with-in group. But I think that's
the trade-off and that's not default. So those who don't require ioprio
stuff working with-in group and those who know that they have got
faster storage will set slice_idle=0. For rest of the SATA users default
is still slice_idle=8.

So I am just creating a path so that high end storage users don't suffer.
Previously they could easily switch to deadline. But because they also
want to use IO control feature and deadline does not support that, we
need to create some paths in CFQ so that it gives deadline like
performance but also provides IO control facility.

Because storage behavior varies so much, typically idling works very well
on single SATA disks and with software RAIDs of SATA disks. But in my
testing deadline is outerforming CFQ on HBA based hardware RAID and
storage arrays. Now ideal thing to address this situation would be some
kind of auto tuning where CFQ realizes the capability of LUN it is
operating on and changes behavior automatically.

But getting auto-tuning is hard, I am trying to address the issue with
the help of tunable in a static manner. So if an admin knows the storage
configuration, he can change the tunables statically. Once a realibale
auto tuning infrastructure is in, we can get rid of these static paths.

Thanks
Vivek

> This is the crux of my issues with CFQ.  It works really well for SATA
> disks.  Once you try running it on enterprise storage, it falls flat.
> Is it a design goal of CFQ to get it to run well on enterprise storage?
> 
> Jens?
> 
> Cheers,
> Jeff
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/