linux-kernel - How do I self diagnose slow write to HDD performance issue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 21 Oct 2008 14:34:01 -0500
From:	"Jonathan Johnson" <johnsonn@...c.edu>
To:	<linux-kernel@...r.kernel.org>
Subject: How do I self diagnose slow write to HDD performance issue

Hello all,

Please CC response to johnsonn@...c.edu 
The only archive of the mailing list I could find was totally empty, and so was the digest.

 I am new to troubleshooting source code in linux.  
I want to collect all the necessary data and analyze as much as possible my self.
When/If that fails I would like to know where and what diagnostic data to send, and to who.

My problem is my RAID 6 is way sloooooooooooooooooooooooooowwww.  Originally, I had
a 6 drive RAID 6 (1000gb,750gb,500gb,500gb,500gb,500gb), and it wrote at about 20mb/s which
for a RAID 6 is junk.  A friend told me my drives were lame so I replaced them with Qty 4 1TB
seagate ST31000340NS drives.  bonnie++ (with only the -d option) used to complete in less than
2 hours and now it barely even gets started in 3 hours. 

I had setup a new array and migrate the data back to new drives which took 3 days(for 1.41TB) or more.
Before I setup the array I tested the SATA 2 drives connected to the mobo SATA connect and got approx 60mb/s
write and even better read speeds per drive.  I had the privilege of borrowing (had to return) a Dell MD3000
and connected that to my controller directly and got vastly better speeds, on 1 drive.

So the drives are fast, and the controller seems to be fast, but when combined really slow.  I have an
adaptec 3805 using aacraid driver.  I am using kernel 2.6.27.2. SuSE 11.0
AMD XP 4200+
2048MB RAM
Intel e1000e (pcie 1x)
ATI Radeon X1300 (pcie 16x)
2 x ATI HDTV Wonders (pci)
Sound Blaster Audigy 4 (pci)
Fusion HDTV Dual Tuner (pcie 1x)
Adaptec 3805 (pcie 4x)

What tools do I run to locate the bottleneck, what am I looking for?
I have used vmstat and top and they report 48-50% "wa" when writing but does not specify which device has it.
It could be the RAID or the internal drive OS.
Does someone specialize in HDD I/O and should I sent it to them.
I have installed Webmin with "Historic Performance module" and have many(12 categories w/ hourly/daily/monthly
 and yearly graphs each category) graphs I can send.

Network I/O is also slow, but that could be related to the disk I/O.

Later,
Jonathan

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/