linux-ext4 - Re: [RFC/PATCH 0/2] ext4: Transparent Decompression Support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130725175307.GA15590@logfs.org>
Date:	Thu, 25 Jul 2013 13:53:08 -0400
From:	Jörn Engel <joern@...fs.org>
To:	Taras Glek <tglek@...illa.com>
Cc:	Dhaval Giani <dgiani@...illa.com>, linux-kernel@...r.kernel.org,
	tytso@....edu, vdjeric@...illa.com, glandium@...illa.com,
	linux-ext4@...r.kernel.org, linux-fsdevel@...r.kernel.org
Subject: Re: [RFC/PATCH 0/2] ext4: Transparent Decompression Support

On Thu, 25 July 2013 09:42:18 -0700, Taras Glek wrote:
> Footprint wins are useful on android, but it's the
> increased IO throughput on crappy storage devices that makes this
> most attractive.

All the world used to be a PC.  Seems to be Android these days.

The biggest problem with compression support in the past was the
physical properties of hard drives (the spinning type, if you can
still remember those).  A random seek is surprisingly expensive, of a
similar cost to 1MB or more of linear read.  So anything that
introduces more random seeks will kill the preciously little
performance you had to begin with.

As long as files are write-once and read-only from that point on, you
can just append a bunch of compressed chunks on the disk and nothing
bad happens.  But if you have a read-write file with random overwrites
somewhere in the middle, those overwrites will change the size of the
compressed data.  You have to free the old physical blocks on disk and
allocate new ones.  In effect, you have auto-fragmentation.

So if you want any kind of support for your approach, I suspect you
should either limit it to write-once files or prepare for a mob of
gray-haired oldtimers with rainbow suspenders complaining about
performance on their antiquated hardware.  And the mob may be larger
than you think.

Jörn

--
Don't worry about people stealing your ideas. If your ideas are any good,
you'll have to ram them down people's throats.
-- Howard Aiken quoted by Ken Iverson quoted by Jim Horning quoted by
   Raph Levien, 1979
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html