linux-ext4 - Re: ext4 backup super block update frequency

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 31 Aug 2018 14:10:08 -0600
From:   Andreas Dilger <adilger@...ger.ca>
To:     Shehbaz Jaffer <shehbazjaffer007@...il.com>
Cc:     linux-ext4@...r.kernel.org
Subject: Re: ext4 backup super block update frequency

On Aug 31, 2018, at 2:00 PM, Shehbaz Jaffer <shehbazjaffer007@...il.com> wrote:
> 
> Thank you for your reply.
> 
> struct ext4_super_block contains various values like s_inodes_count,
> s_free_inodes_count etc. that would vary after each file creation. If
> backup super blocks do not get updated, backup will continue having
> these values same as that at file system creation time.
> 
> If the primary super block is no longer available due to
> corruption/bad cable, HBA error, does ext4 rely on the updated super
> block value(s) recorded in journal to restore the primary super block?

No, these transient values are only estimates, and do not affect the
correctness of the filesystem.  In fact, ext4 will not even update
them in most cases except a clean unmount, since this causes unnecessary
contention in the block allocation code.  e2fsck will fix these values
without considering it an error.

The kernel keeps up-to-date values in memory, and the actual counters
used are kept per block group, so that parallel (de)allocation can be
done in different block groups.

Cheers, Andreas


> On Fri, Aug 31, 2018 at 3:21 PM Andreas Dilger <adilger@...ger.ca> wrote:
>> 
>> On Aug 31, 2018, at 10:40 AM, Shehbaz Jaffer <shehbazjaffer007@...il.com> wrote:
>>> 
>>> Hi All,
>>> 
>>> I am trying to study how backup superblocks are updated and used in
>>> the ext4 file system. I create a 2GB file system. This creates 5
>>> backup superblocks with the sparse sb option. I then check the diff of
>>> the backup blocks before and after multiple mount, write(2) fsync(2)
>>> and close(2) and unmount operations.
>>> 
>>> I can see the primary superblock get updated but I do not see the
>>> backup super blocks being updated. My intuition is that the backup
>>> blocks are only present so that the recovery can be done by replaying
>>> the journal on the backup superblock. Are they updated each time the
>>> journal gets full so that the "refreshed" journal can now be replayed
>>> on updated backup superblock in case of a crash?
>>> If this is incorrect, at what frequency do backup superblocks updated?
>>> 
>>> If we compare this behavior with BtrFS, I can see that for each update
>>> on fs tree, the primary block (at offset 64KB) and backup superblock
>>> (at 64MB) gets updated.
>> 
>> The backup ext4 superblocks are never updated by the kernel, only after
>> a successful e2fsck, tune2fs, resize2fs, or other userspace operation.
>> 
>> This avoids clobbering the backups with bad data if the kernel has a bug
>> or device error (e.g. bad cable, HBA, etc).
>> 
>> Cheers, Andreas
>> 
>> 
>> 
>> 
>> 
> 
> 
> --
> Shehbaz Jaffer


Cheers, Andreas






Download attachment "signature.asc" of type "application/pgp-signature" (874 bytes)