linux-ext4 - Re: [PATCH] Add better example about how to compress e2image raw image

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4E80A78C.4050301@redhat.com>
Date:	Mon, 26 Sep 2011 11:25:48 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Bernd Schubert <bernd.schubert@...m.fraunhofer.de>
CC:	"Ted Ts'o" <tytso@....edu>, Carlos Maiolino <cmaiolino@...hat.com>,
	Andreas Dilger <adilger@...ger.ca>, linux-ext4@...r.kernel.org
Subject: Re: [PATCH] Add better example about how to compress e2image raw
 image

On 9/26/11 7:22 AM, Bernd Schubert wrote:
> On 09/24/2011 06:31 PM, Ted Ts'o wrote:
>> On Fri, Sep 23, 2011 at 05:51:24PM -0300, Carlos Maiolino wrote:
>>> On Fri, Sep 23, 2011 at 02:24:23PM -0600, Andreas Dilger wrote:
>>>> On 2011-09-23, at 12:47 PM, Carlos Maiolino wrote:
>>>>> The current example in the man page uses bzip2 to compress
>>>>> the raw image file created by the e2image, but, bzip2 does
>>>>> not honors sparse files, which causes the image to have the
>>>>> same size of the filesystem.
>>>>> Using tar together with bzip2 will make the compressed file
>>>>> to honor the sparsed file, which makes it more transportable
>>>>> than the current one if the filesystem is large.
>>
>> The problem with using tar is that it requires extra disk space by the
>> user --- somewhere a bit more than double the extra disk space
>> (because you need to have space for the hda1.e2i file before it gets
>> compressed).  For very large file systems, this can be quite
>> significant.  My general philosophy has been to make things easy as
>> possible for the users as being more important for the developers.
>>
>> For the developers, we do have contrib/make-sparse.c.  All we have to do is:
>>
>>      bunzip2<  hda1.e2i.bz2 | make-sparse hda1.e2i
>>
>> ... and this creates a sparse file in hda1.e2i.
> 
> The problem is that the bzip2 run will take a huge amount of time to
> compress all the zeros. In 2009 (with a recent CPU of that time) I
> aborted such a run for a 8TiB file system after a couple of days,
> then stored the e2image directly on disk and compressed it with tar
> and sparse support, which finished after only 12 hours... I don't
> think more modern CPUs are much faster for single threaded runs as
> bzip2 does it. So IMHO the man page should at least warn about that
> issue and suggest to use a similar tar command.

Agreed!

I think they both have their place.

passing images around in qcow format may be best in the long run though.

-Eric

> 
> Cheers,
> Bernd
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html