linux-ext4 - Re: Create ext2 filesystem from a directory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190220005515.GD12500@redhat.com>
Date:   Wed, 20 Feb 2019 00:55:15 +0000
From:   "Richard W.M. Jones" <rjones@...hat.com>
To:     "Theodore Y. Ts'o" <tytso@....edu>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Create ext2 filesystem from a directory

On Tue, Feb 19, 2019 at 07:44:16PM -0500, Theodore Y. Ts'o wrote:
> On Tue, Feb 19, 2019 at 08:03:33AM +0000, Richard W.M. Jones wrote:
> > 
> > This might interest/disgust/shock/scare(?!) people on this list:
> > 
> > $ time ./nbdkit --filter=partition -U - linuxdisk . partition=1 --run 'qemu-img convert $nbd /var/tmp/ext2fs.img'
> > 
> 
> For those of us who aren't really familiar with nbdkit, what does this do?

There's quite a lot packed into a small space ...

Firstly nbdkit is an NBD server which is unique in having plugins to
implement different data sources.  In this case I have written a
plugin called "linuxdisk" which creates a GPT-partitioned
ext2-formatted disk image from a local directory ("." in this case).

nbdkit can serve over IP or Unix domain sockets, including randomly
created Unix domain sockets ("-U -").

nbdkit can also run other commands and exit when those commands have
finished (we call this "captive nbdkit").  In this case we ask nbdkit
to run a qemu-img command and then exit when qemu-img finishes.

"qemu-img convert $nbd /var/tmp/ext2fs.img" connects to the private
Unix domain socket (nbdkit sets $nbd to a suitable string to make this
happen).  It then copies the data out of that NBD socket to a local
file, optionally doing a format conversion although not in this case.

Finally nbdkit has a concept of filters which can be placed on top of
plugins to select parts of the data.  Remember that the "linuxdisk"
plugin creates GPT-partitioned ext2 disk images.  However we want only
the naked filesystem.  Therefore we place the partition filter on top
to pick out and serve only the partition content, which is the naked
ext2 filesystem.

I recently did a talk about nbdkit if you want to find out more:
https://rwmj.wordpress.com/2019/02/04/video-take-your-loop-mounts-to-the-next-level-with-nbdkit/

Here's another fun thing you can do with the linuxdisk plugin:
https://rwmj.wordpress.com/2019/02/19/nbdkit-linuxdisk-plugin/

Hope that helps!

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top