linux-ext4 - Re: Filesystem size problem.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 13 Dec 2016 13:48:21 -0700
From:   Andreas Dilger <adilger@...ger.ca>
To:     Simon Matthews <simon.d.matthews@...il.com>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: Filesystem size problem.

On Dec 12, 2016, at 7:48 PM, Simon Matthews <simon.d.matthews@...il.com> wrote:
> 
> On Mon, Dec 12, 2016 at 2:36 PM, Andreas Dilger <adilger@...ger.ca> wrote:
>> On Dec 9, 2016, at 9:35 PM, Eric Sandeen <sandeen@...hat.com> wrote:
>>> 
>>> On 12/9/16 2:29 PM, Andreas Dilger wrote:
>>>> On Dec 8, 2016, at 10:40 PM, Simon Matthews <simon.d.matthews@...il.com> wrote:
>>>>> 
>>>>> I have an ext3 filesystem that will not mount under newer versions of
>>>>> the kernel and I hope someone here can help.
>>>>> 
>>>>> Obviously, one solution is "backup and re-create from scratch". I have
>>>>> the backups, but I hope that there may be a quicker method to fix the
>>>>> issues.
>>>>> 
>>>>> The root issue is that the filesystem is very slightly smaller than
>>>>> the allocated space.
>>> 
>>> So the device is now smaller than the filesystem thinks, right?
>>> 
>>>>> The filesystem exists on a MDRAID device and I think that when I
>>>>> converted the MDRAID to a newer metadata version, it truncated the
>>>>> available size, slightly. However, how I got here isn't really
>>>>> important, fixing it now is.
>>>> 
>>>> Running "e2fsck -fy" should fix this.  I'd recommend to use the latest
>>>> version of e2fsck.
>>> 
>>> Reaslly?  e2fsck can change total blocks in the superblock to accomodate
>>> a shrunken device?  That's a new one for me...
>> 
>> Strange, I thought this case was handled properly by e2fsck.
>> 
>> You could probably fix this with:
>> 
>> # debugfs -w -R "ssv blocks_count 693359326" /dev/md5
> 
> 
> "probably"?
> 
> How safe or dangerous is this? Does the filesystem have to be unmounted first?

The filesystem *definitely* needs to be unmounted first.

I wouldn't classify this change as being super dangerous, because it is
only removing a few blocks from the end of the filesystem, and e2fsck
should handle the case where inodes reference blocks beyond EOFS as any
other corrupt blocks.  I don't think that is likely to happen in this
case, unless your filesystem is extremely full, since extN filesystems
front-end bias allocations to the faster part of the storage device.

That said, I haven't tested this process[*], and if you are concerned that
it may eat your data (that is always possible) you should make a backup.
You should probably make a backup even if you aren't going to do this, as
that is always a good idea.  As with any free advice you on the internet
YMMV, and the final decision is up to you.

The other option is to make a new filesystem on a second set of storage
and then copy the old files over.  That also has benefits that the old
filesystem acts as your backup, you get any new features enabled in ext4
when the filesystem is newly formatted, and the files will likely be laid
laid out on disk contiguously during the copy, so it will defragment the
filesystem (not that ext4 needs this very much).

PS: I added "-w" to the debugfs command above, or it would have failed

Cheers, Andreas

[*] I did just try this on a test filesystem and it worked OK for me:

[root@...kie ~]# dumpe2fs -h /dev/dm-53 | grep "Block count"
dumpe2fs 1.42.13.wc5 (15-Apr-2016)
Block count:              2621440
[root@...kie ~]# debugfs -w -R "ssv blocks_count 2621400" /dev/dm-53
debugfs 1.42.13.wc5 (15-Apr-2016)
[root@...kie ~]# e2fsck -fn /dev/dm-53
e2fsck 1.42.13.wc5 (15-Apr-2016)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Free blocks count wrong for group #79 (26047, counted=26007).
Fix? no

Free blocks count wrong (2226761, counted=2226721).
Fix? no

Padding at end of block bitmap is not set. Fix? no

myth_2-MDT0000: ********** WARNING: Filesystem still has errors **********

myth_2-MDT0000: 1010990/2621440 files (0.1% non-contiguous), 394639/2621400 blocks
[root@...kie ~]# e2fsck -fp /dev/dm-53
myth_2-MDT0000: Padding at end of block bitmap is not set. FIXED.
myth_2-MDT0000: 1010990/2621440 files (0.1% non-contiguous), 394679/2621400 blocks

Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)