linux-ext4 - Re: sparsify - utility to punch out blocks of 0s in a file

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4F301E8B.7050909@oracle.com>
Date:	Mon, 06 Feb 2012 10:40:11 -0800
From:	Sunil Mushran <sunil.mushran@...cle.com>
To:	Eric Sandeen <sandeen@...hat.com>
CC:	ext4 development <linux-ext4@...r.kernel.org>,
	xfs-oss <xfs@....sgi.com>, ocfs2-devel@....oracle.com
Subject: Re: sparsify - utility to punch out blocks of 0s in a file

On 02/04/2012 12:04 PM, Eric Sandeen wrote:
> Now that ext4, xfs,&  ocfs2 can support punch hole, a tool to
> "re-sparsify" a file by punching out ranges of 0s might be in order.
>
> I whipped this up fast, it probably has bugs&  off-by-ones but thought
> I'd send it out.  It's not terribly efficient doing 4k reads by default
> I suppose.
>
> I'll see if util-linux wants it after it gets beat into shape.
> (or did a tool like this already exist and I missed it?)
>
> (Another mode which does a file copy, possibly from stdin
> might be good, like e2fsprogs/contrib/make-sparse.c ?  Although
> that can be hacked up with cp already).
>
> It works like this:
>
> [root@...de sparsify]# ./sparsify  -h
> Usage: sparsify [-m min hole size] [-o offset] [-l length] filename

So I have a similar tool queued up in ocfs2-tools. Named puncher.
http://oss.oracle.com/git/?p=ocfs2-tools.git;a=shortlog;h=puncher

I'll pull it out if we get something in util-linux. But maybe you can 
extract something useful from it.

Like.... maybe doing dry-run as default. It is an inplace modification
after all. Also using a large hole size as default (1MB). Over using 
hole punching will negatively affect read performance. We should make 
the sane choice for the user.

On a related note, it may make sense for ext4 to populate the cluster 
size (bigalloc) in stat.st_blksize.

2 cents...
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html