linux-ext4 - [LSF/MM TOPIC] Use generic FS in virtual environments challenges and solutions

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 29 Jan 2014 18:32:58 +0400
From:	Dmitry Monakhov <dmonakhov@...nvz.org>
To:	lsf-pc@...ts.linux-foundation.org
CC:	linux-fsdevel@...r.kernel.org, linux-ext4@...r.kernel.org,
	Pavel Emelianov <xemul@...allels.com>,
	Konstantin Khorenko <khorenko@...allels.com>
Subject: [LSF/MM TOPIC] Use generic FS in virtual environments challenges and solutions

Number of virtual environment/container solutions are grow rapidly, here is just
small list of well known names (qemu/kvm, VMware, openvz, LXC, etc)
There are two main challenges any VE solution should overcome:
1) Minimize Guest OS modification (ideally run unmodified binaries)
2) Resource sharing between several VE contexts (mem,cpu,disk)
There are plenty of advanced algorithms for CPU and memory sharing
between VEs. There are no many effective virtualization schemes for
disk at the moment.

OpenVZ project has interesting experience in fs/disk virtualization.
I want to propose three topics about fs/disk virtualization:

1) Effective space allocation scheme aka "Thin provision" [1]
   Generic filesystem tries to spawn all it's data across whole disk.
   In case of virtual images this result continuous VImage growth
   during FS activity even if actual FS disk usage is low.

   We have done some research and modified ext4 block allocator
   which allow us to reduce VImage swelling effect, I would like to
   discuss our finding's.

2) Space reclamation FS/disk shrinking
   FS/disk growth is relatively simple operation most disk images and FS allow
   online grow [2], but shrink is very heavyweight operation. I would like
   to discuss some tricks how to make offline/online shrink less intrusive.

3) Filesystem error detection and correction
   At this moment most filesystem may detect internal errors and perform
   basic actions(panic,remount_ro) but this reaction is not suitable
   for virtual environment because HardwareNode should continue to
   operate and fix dedicated VE as soon as possible.
   For this purpose it is reasonable to:
   A) Implement fs event notification API similar to UEVENTs for devices or
      quota event API. I would like to discuss this API.
   B) Reduce fsck time. Theodore Tso have announced initiative to implement
      ffck for ext4 [3]. I want to discuss perspectives of design and
      implementation online fsck for ext4.

Footnotes: 
[1]  http://en.wikipedia.org/wiki/Thin_provisioning

[2]  http://openvz.org/Ploop

[3]  http://marc.info/?l=linux-ext4&m=138661211607779&w=2

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html