linux-ext4 - Re: New resize interface implementation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAOQ4uxg1JeyhpR4i5Krt0RJs5txpa+kKrMoBbU_n26Gzq+xkFw@mail.gmail.com>
Date:	Thu, 1 Sep 2011 14:30:03 +0200
From:	Amir Goldstein <amir73il@...il.com>
To:	Justin Maggard <jmaggard10@...il.com>
Cc:	ext4 development <linux-ext4@...r.kernel.org>,
	Yongqiang Yang <xiaoqiangnk@...il.com>
Subject: Re: New resize interface implementation

On Fri, Aug 26, 2011 at 10:06 PM, Justin Maggard <jmaggard10@...il.com> wrote:
> On Wed, Aug 17, 2011 at 12:28 AM, Yongqiang Yang <xiaoqiangnk@...il.com> wrote:
>> On Tue, Aug 16, 2011 at 8:22 AM, Justin Maggard <jmaggard10@...il.com> wrote:
>> > Does this patch set combined with your e2fsprogs patch add 64-bit resize
>> > support now, or does it just make it easier to add later?
>> YES. e2fsprgos's patch is ready too.
>
> So I finally got around to gather the hardware and patching all the
> software components to try out this 64-bit expansion code.  The first
> thing I noticed is that there is still a check to make sure the block
> count is 32 bits.  However, I can get around it by specifying a size
> string (something like "20T") rather than a block count, in which case
> it will actually try the expansion.

The fact that you can get around this check is a bug.
As you have observed, things won't be pretty if you try to resize over 16TB
using resize inode and I don't think it is intended to work.


>
>> > If I am making a 64-bit ext4 filesystem today (20TB), and hoping to resize
>> > it next year to 30TB what features should I set?  In my searching it sounded
>> > like maybe I would need meta_bg, but it is not compatible with the default
>> > resize_inode.
>> You can understand meta_bg here http://linuxsoftware.co.nz/wiki/ext4.
>> Now, ext4 with meta_bg does not support resize.  It is in ext4's TODO list.
>> The feature you should set is resize_inode.
>>
>> > Also, if I am making a <16TB filesystem today, should I turn on the 64-bit
>> > flag in order to expand to >16TB in the future?
>> Yes.  You should turn on 64 bit feature.  If the block number is 32
>> bit, the size it can support is 2^32 * 2^(log blocksize),  4K
>> blocksize as an example, it maximum size of a filesystem is 2^32 *
>> 2^12 = 2^44 = 16TB.
>
> I think this is where the real problem is with this 64-bit resize
> support.  With the 64-bit flag set, the most I can expand by online is
> just 8TB over the life of the filesystem, because my reserved GDT
> blocks get used up twice as fast as with a 32-bit filesystem.  Is
> there any way around this?

The maximum reserved GDT blocks is EXT2_ADDR_PER_BLOCK(sb),
which is 1024 by default, just enough for expanding the 64-bit fs by 8TB,
as you have observed.
But also, resize inode cannot store 64-bit block addresses of GDT backups
beyond 16TB, so your fs (resize inode in particular) are most likely corrupted.

There is no point in getting around these issues.
We should get on top of them and implement online resize of meta_bg.
If your intention is to create a 20TB fs today and resize it in the
(not very near) future
then you should probably use meta_bg instead of resize_inode.

Amir.
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html