linux-kernel - Re: Linux 2.6.29

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <49D56DF6.5020300@garzik.org>
Date:	Thu, 02 Apr 2009 22:01:26 -0400
From:	Jeff Garzik <jeff@...zik.org>
To:	Linus Torvalds <torvalds@...ux-foundation.org>
CC:	Andrew Morton <akpm@...ux-foundation.org>,
	David Rees <drees76@...il.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: Linux 2.6.29

Linus Torvalds wrote:
> Feel free to give it a try. It _should_ maintain good write speed while 
> not disturbing the system much. But I bet if you added the "fadvise()" it 
> would disturb things even _less_.
> 
> My only point is really that you _can_ do streaming writes well, but at 
> the same time I do think the kernel makes it too hard to do it with 
> "simple" applications. I'd love to get the same kind of high-speed 
> streaming behavior by just doing a simple "dd if=/dev/zero of=bigfile"
> 
> And I really think we should be able to.
> 
> And no, we clearly are _not_ able to do that now. I just tried with "dd", 
> and created a 1.7G file that way, and it was stuttering - even with my 
> nice SSD setup. I'm in my MUA writing this email (obviously), and in the 
> middle it just totally hung for about half a minute - because it was 
> obviously doing some fsync() for temporary saving etc while the "sync" was 
> going on.
> 
> With the "overwrite.c" thing, I do get short pauses when my MUA does 
> something, but they are not the kind of "oops, everything hung for several 
> seconds" kind. 

Attached is my slightly-modified version of overwrite.c, modded to bound 
the file size and to use fadvise().

On a 128GB, 3.0 Gbps no-name SATA SSD, x86-64, ext3, 2.6.29 vanilla kernel:

+ ./overwrite -b 3000 /spare/tmp/test.dat
writing 3000 buffers of size 8m
       23.429 GB written in 1019.25 (23 MB/s)

real	17m0.211s
user	0m0.028s
sys	1m5.800s


+ ./overwrite -b 3000 -f /spare/tmp/test.dat
using fadvise()
writing 3000 buffers of size 8m
       23.429 GB written in 1060.54 (22 MB/s)

real	17m41.446s
user	0m0.036s
sys	1m9.016s


The most interesting thing I found:  the SSD does 80 MB/s for the first 
~1 GB or so, then slows down dramatically.  After ~2GB, it is down to 32 
MB/s.  After ~4GB, it reaches a steady speed around 23 MB/s.


--------------------------------------------------

On a 500GB, 3.0Gbps Seagate SATA drive, x86-64, ext3, 2.6.29 vanilla kernel:

+ ./overwrite -b 3000 /garz/tmp/test.dat
writing 3000 buffers of size 8m
       23.429 GB written in 539.06 (44 MB/s)

real	9m0.348s
user	0m0.064s
sys	1m2.704s


+ ./overwrite -b 3000 -f /garz/tmp/test.dat
using fadvise()
writing 3000 buffers of size 8m
       23.429 GB written in 535.08 (44 MB/s)

real	8m55.971s
user	0m0.044s
sys	1m6.600s


There is a similar performance fall-off for the Seagate, but much less 
pronounced:
	After 1GB:	52 MB/s
	After 2GB:	44 MB/s
	After 3GB:	steady state




There appears to be a small increase in system time with "-f" (use 
fadvise), but I'm guessing time(1) does not really give a good picture 
of overall system time used, when you include background VM activity.

	Jeff




View attachment "overwrite.c" of type "text/plain" (2188 bytes)