linux-kernel - RE: CCISS performance drop in buffered disk reads in newer kernels

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <0F5B06BAB751E047AB5C87D1F77A77886994120A79@GVW0547EXC.americas.hpqcorp.net>
Date:	Mon, 7 Dec 2009 19:30:37 +0000
From:	"Miller, Mike (OS Dev)" <Mike.Miller@...com>
To:	Jens Axboe <jens.axboe@...cle.com>,
	Ozan Ça??layan <ozan@...dus.org.tr>
CC:	linux-kernel <linux-kernel@...r.kernel.org>,
	"scameron@...rdog.cce.hp.com" <scameron@...rdog.cce.hp.com>
Subject: RE: CCISS performance drop in buffered disk reads in newer kernels

 

> -----Original Message-----
> From: Jens Axboe [mailto:jens.axboe@...cle.com] 
> Sent: Monday, December 07, 2009 12:40 PM
> To: Ozan Ça??layan
> Cc: Miller, Mike (OS Dev); linux-kernel; scameron@...rdog.cce.hp.com
> Subject: Re: CCISS performance drop in buffered disk reads in 
> newer kernels
> 
> On Mon, Dec 07 2009, Ozan Ça??layan wrote:
> > Miller, Mike (OS Dev) wrote:
> > > Ozan,
> > > I'm aware of the performance drop. Please see: 
> http://bugzilla.kernel.org/show_bug.cgi?id=13127. I removed 
> the huge read ahead value of 1024 that we used because users 
> were complaining about small writes being starved. That was 
> back around the 2.6.25 timeframe. Since that timeframe there 
> have no changes in the main i/o path. I'll get back on this 
> as time allows.
> > >
> > > Meanwhile, you can tweak some of the block layer tunables as such.
> > >
> > >     echo 64 > /sys/block/cciss\!c0d1/queue/read_ahead_kb
> > > OR
> > >     blockdev --setra 128 /dev/cciss/c0d1
> > >
> > > These are just example values. There is also 
> max_hw_sectors_kb and max_sectors_kb that be adjusted.
> > >   
> > 
> > Hi,
> > 
> > Actually the "#define READ_AHEAD 1024" was removed on March 
> 2008 which 
> > was included in the 2.6.25.y tree so 2.6.25.20 has 128kB read_ahead 
> > value too.
> > 
> > *But* setting read_ahead to 2048 increases buffered disk 
> read average 
> > from 60~MB/s to 190~MB/s hence the kernel compile time 
> drops to 2 minutes.
> > 
> > So maybe the regression/change is in another place?
> > 
> > The server is just a compile-farm so it's triggered by 
> hand, compiles 
> > distribution's packages and stays idle until the next 
> compilation queue.
> > Is it safe/OK to use that 2048kB read_ahead value for such workload?
> 
> Yes, it's definitely safe.

I agree. 

> 
> > (max_hw_sectors_kb is 512 on my 2.6.25.20 setup and 1024 on 
> 2.6.30.9 
> > but it seems that it's read-only)
> 
> The *_hw_* values are the driver exported hardware limits, so 
> they are always read-only.

Ahhh, I didn't know that. There is also an nr_requests attribute which to me implies limiting requests somewhere. The value of nr_request is 128 but the max commands to the cciss controllers exceed that value. What is nr_request supposed to do?

-- mikem

> 
> --
> Jens Axboe
> 
> --
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/