linux-kernel - Re: [PATCH][RFC] Complex filesystem operations: split and join

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <201006171704.14541.hka@qbs.com.pl>
Date:	Thu, 17 Jun 2010 17:04:13 +0200
From:	Hubert Kario <hka@....com.pl>
To:	"David Pottage" <david@...ctric-spoon.com>
Cc:	"Nikanth Karthikesan" <knikanth@...ell.com>,
	"OGAWA Hirofumi" <hirofumi@...l.parknet.co.jp>,
	linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org,
	"Alexander Viro" <viro@...iv.linux.org.uk>,
	"Christoph Hellwig" <hch@....de>,
	"Chris Mason" <chris.mason@...cle.com>, linux-btrfs@...r.kernel.org
Subject: Re: [PATCH][RFC] Complex filesystem operations: split and join

On Tuesday 15 June 2010 17:16:06 David Pottage wrote:
> On 15/06/10 11:41, Nikanth Karthikesan wrote:
> > I had a one-off use-case, where I had no free-space, which made me
> > think along this line.
> > 
> > 1. We have the GNU split tool for example, which I guess, many of us
> > use to split larger files to be transfered via smaller thumb drives,
> > for example. We do cat many files into one, afterwards. [For this
> > usecase, one can simply dd with seek and skip and avoid split/cat
> > completely, but we dont.]
> 
> I am not sure how you gain here as either way you have to do I/O to get
> the split files on and off the thumb drive. It might make sense if the
> thumb drive is formated with btrfs, and the file needs to be copied to
> another filling system that can't handle large files (eg FAT-16), but I
> would say that is unlikely.
> 

But you do have to do only half as much of I/O with those features 
implemented.

The old way is:
1. Have a file
2. split a file (in effect use twice as much drive space)
3. copy fragments to flash disks

The btrfs way would be:
1. Have a file
2. split the file by using COW and referencing blocks in the original file (in 
effect using only a little more space after splitting)
3. copy fragments to flash disks

the amount of I/O in the second case is limited only to metadata operations, 
in the first case, all data must be duplicated

-- 
Hubert Kario
QBS - Quality Business Software
02-656 Warszawa, ul. Ksawerów 30/85
tel. +48 (22) 646-61-51, 646-74-24
www.qbs.com.pl

System Zarządzania Jakością
zgodny z normą ISO 9001:2000
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/