lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Wed, 09 Jun 2010 10:43:37 +0400 From: Michael Tokarev <mjt@....msk.ru> To: Dave Chinner <david@...morbit.com> CC: Linux-kernel <linux-kernel@...r.kernel.org>, xfs@....sgi.com Subject: Re: xfs, 2.6.27=>.32 sync write 10 times slowdown [was: xfs, aacraid 2.6.27 => 2.6.32 results in 6 times slowdown] 09.06.2010 03:18, Dave Chinner wrote: > On Wed, Jun 09, 2010 at 12:34:00AM +0400, Michael Tokarev wrote: [] >> Simple test doing random reads or writes of 4k blocks in a 1Gb >> file located on an xfs filesystem, Mb/sec: >> >> sync direct >> read write write >> 2.6.27 xfs 1.17 3.69 3.80 >> 2.6.32 xfs 1.26 0.52 5.10 >> ^^^^ >> 2.6.32 ext3 1.19 4.91 5.02 >> >> Note the 10 times difference between O_SYNC and O_DIRECT writes >> in 2.6.32. This is, well, huge difference, and this is where >> the original slowdown comes from, apparently. > > Are you running on the raw block device, or on top of LVM/DM/MD to > split up the space on the RAID drive? DM+MD have grown barrier > support since 2.6.27, so it may be that barriers are now being > passed down to the raid hardware on 2.6.32 and they never were on > 2.6.27. Can you paste the output of dmesg when the XFS filesystem in That's why I asked how to tell if barriers are actually hitting the device in question. No, this is the only machine where DM/MD is _not_ used. On all other machines we use MD software raid, this machine comes with an onboard raid controller that does not work in JBOD mode so I weren't able to use linux software raid. This is XFS on top of Adaptec RAID card, nothing in-between. Also, as I mentioned in the previous email, remounting with nobarrier makes no difference whatsoever. (Another side note here - I discovered that unknown options are silently ignored in "remount mode" while correctly rejected in "plain mount" mode, -- it looks like a kernel bug actually, but it's entirely different issue). > question is mounted on both 2.6.27 and 2.6.32 so we can see if > there is a difference in the use of barriers? > > Also, remember that O_DIRECT does not imply O_SYNC. O_DIRECT writes > only write data, while O_SYNC will also write metadata and/or the > log. I know this. I also found osyncisosync and osyncisdsync mount options, and when I try to use the latter, kernel tells it's the default and hence deprecated. I don't need metadata updates, but it _looks_ like the system is doing such updates (with barriers or flushes?) anyway even when mounted with -o osyncisdsync it behaves the same: very slow. I also experimented with both O_SYNC|O_DIRECT: it is as slow as without O_DIRECT, i.e. O_SYNC makes whole thing slow regardless of other options. I looked at the dmesg outputs, and there's no relevant differences related to block devices or usage of barriers. For XFS it always mounts like this: SGI XFS with ACLs, security attributes, large block/inode numbers, no debug enabled SGI XFS Quota Management subsystem XFS mounting filesystem sda6 and for the device in question, it is always like Adaptec aacraid driver 1.1-5[2456]-ms aacraid 0000:03:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24 AAC0: kernel 5.1-0[8832] Feb 1 2006 AAC0: monitor 5.1-0[8832] AAC0: bios 5.1-0[8832] AAC0: serial 267BE0 AAC0: Non-DASD support enabled. AAC0: 64bit support enabled. AAC0: 64 Bit DAC enabled scsi0 : aacraid scsi 0:0:0:0: Direct-Access Adaptec f0500 V1.0 PQ: 0 ANSI: 2 sd 0:0:0:0: [sda] 286715904 512-byte hardware sectors (146799 MB) sd 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Mode Sense: 06 00 10 00 sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, supports DPO and FUA sda: sda1 sda2 sda3 < sda5 sda6 > There are tons of other differences, but that is to be expected (like format of CPU topology printing which is changed between .27 and .32). Thanks! /mjt -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@...r.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists