linux-kernel - Re: [ANNOUNCE] Ramback: faster than a speeding bullet

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <200803130012.36763.phillips@phunq.net>
Date:	Wed, 12 Mar 2008 23:12:35 -0800
From:	Daniel Phillips <phillips@...nq.net>
To:	david@...g.hm
Cc:	David Newall <davidn@...idnewall.com>,
	Chris Friesen <cfriesen@...tel.com>,
	Alan Cox <alan@...rguk.ukuu.org.uk>,
	linux-kernel@...r.kernel.org
Subject: Re: [ANNOUNCE] Ramback: faster than a speeding bullet

On Wednesday 12 March 2008 23:32, david@...g.hm wrote:
> looking at the comparison of a 500G filesystem with 500G of ram allocated 
> for a buffer cache.
> 
> yes, initially it will be a bit slower (until the files get into the 
> buffer cache), and if fsync is disabled all writes will go to the buffer 
> cache (until writeout hits)
> 
> I may be able to see room for a few percent difference, but not 2x, let 
> alone 25x.

My test ran 25 times faster because it was write intensive and included
sync.  It did not however include seeks, which can cause an even bigger
performance gap.

The truth is, my system has _more_ cache available for file buffering
than I used for the ramdisk, and almost every file operation I do
(typically dozens of tree diffs, hundreds of compiles per day) goes
_way_ faster on the ram disk.  Really, really a lot faster.  Because
frankly, Linux is not very good at using its file cache these days.
Somebody ought to fix that.  (I am busy fixing other things.)

In other, _real world_ NFS file serving tests, we have seen 20 - 200
times speedup in serving snapshotted volumes via NFS, using ddsnap
for snapshots and replication.  While it is true that ddsnap will
eventually be optimized to improved performance on spinning media,
I seriously doubt it will ever get closer than a factor of 20 or so,
with a typical read/write mix.

But that is just the pragmatic reality of machines everybody has these
days, let us not get too wrapped up in that.  Think about the Violin
box.  How are you going to put 504 gigabytes of data in buffer cache?
Tell me how a transaction processing system is going to run with
latency measured in microseconds, backed by hard disk, ever?

Really guys, ramdisks are fast.  Admit it, they are really really fast.
So I provide a way to make them persistent also.  For free, I might
add.

Why am I reminded of old arguments like "if men were meant to fly, God
would have given them wings"?  Please just give me your microsecond
scale transaction processing solution and I will be impressed and
grateful.  Until then... here is mine.  Service with a smile.

Daniel
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/