lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <171484520952.2626447.2160419274451668597@localhost>
Date: Sat, 04 May 2024 19:53:29 +0200
From: Johannes Schauer Marin Rodrigues <josch@...ter-muffin.de>
To: linux-ext4@...r.kernel.org
Subject: Re: created ext4 disk image differs depending on the underlying filesystem

Quoting Johannes Schauer Marin Rodrigues (2024-05-04 16:32:50)
> I originally observed this issue when creating ext4 disk images on a 9p
> filesystem which differed from the images I created on a tmpfs. I observed
> that the difference also exists when the underlying file system is fat32, so
> I'm using this as an example here. For what it's worth, the ext4 filesystem
> images created on a tmpfs are identical to those created on an ext4 fs. To
> demonstrate the issue, please see the script at the end of this mail (it
> requires sudo to mount and unmount the fat32 disk image). As you can see from
> the printed hashes, the disk images produced outside the fat32 disk are
> always identical as expected. The diff between the reproducible images and
> those stored on fat32 is also very short but I don't know what data is stored
> at those points:
> 
> @@ -85,7 +85,7 @@
>  00000540: 0000 0000 0000 0000 0000 0000 0000 1000  ................
>  00000550: 0000 0000 0000 0000 0000 0000 2000 2000  ............ . .
>  00000560: 0200 0000 0000 0000 0000 0000 0000 0000  ................
> -00000570: 0000 0000 0401 0000 8c04 0000 0000 0000  ................
> +00000570: 0000 0000 0401 0000 4900 0000 0000 0000  ........I.......
>  00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> @@ -125,9 +125,9 @@
>  000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> -000007f0: 0000 0000 0000 0000 0000 0000 264c 0251  ............&L.Q
> +000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5  ............d...
>  00000800: 1200 0000 2200 0000 3200 0000 9d03 7300  ...."...2.....s.
> -00000810: 0200 0000 0000 0000 babb 8a41 7300 2004  ...........As. .
> +00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5  ...........As.i.
>  00000820: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  00000830: 0000 0000 0000 0000 bc7a 6e31 0000 0000  .........zn1....
>  00000840: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> 
> Any idea what is going on? Is there a better way to diff two ext4 disk images
> than diffing the xxd output? If I try diffing the dumpe2fs output I get these
> differences:
> 
> @@ -32,7 +32,7 @@
>  Maximum mount count:      -1
>  Last checked:             Fri May  3 16:14:49 2024
>  Check interval:           0 (<none>)
> -Lifetime writes:          1164 kB
> +Lifetime writes:          73 kB
>  Reserved blocks uid:      0 (user root)
>  Reserved blocks gid:      0 (group root)
>  First inode:              11
> @@ -44,7 +44,7 @@
>  Directory Hash Seed:      0b7f9cfd-0113-486c-a453-4f5483bd486b
>  Journal backup:           inode blocks
>  Checksum type:            crc32c
> -Checksum:                 0x51024c26
> +Checksum:                 0xa5bbca64
>  Checksum seed:            0xf81d767d
>  Orphan file inode:        12
>  Journal features:         (none)
> @@ -56,7 +56,7 @@
>  Journal start:            0
>  
>  
> -Group 0: (Blocks 1-2047) csum 0x0420
> +Group 0: (Blocks 1-2047) csum 0xf569 [ITABLE_ZEROED]
>    Primary superblock at 1, Group descriptors at 2-2
>    Reserved GDT blocks at 3-17
>    Block bitmap at 18 (+17), csum 0x7abcbbba
> 
> Why would these bits differ depending on the filesystem on which the disk image
> is stored? Is there a way to equalize this information so that the disk image
> looks the same independent on the underlying filesystem?

The diff becomes a bit smaller when using
-E lazy_itable_init=0,assume_storage_prezeroed=0,nodiscard

@@ -85,7 +85,7 @@
 00000540: 0000 0000 0000 0000 0000 0000 0000 1000  ................
 00000550: 0000 0000 0000 0000 0000 0000 2000 2000  ............ . .
 00000560: 0200 0000 0000 0000 0000 0000 0000 0000  ................
-00000570: 0000 0000 0401 0000 ac04 0000 0000 0000  ................
+00000570: 0000 0000 0401 0000 4900 0000 0000 0000  ........I.......
 00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
@@ -125,7 +125,7 @@
 000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-000007f0: 0000 0000 0000 0000 0000 0000 fb8d a90f  ................
+000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5  ............d...
 00000800: 1200 0000 2200 0000 3200 0000 9d03 7300  ...."...2.....s.
 00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5  ...........As.i.
 00000820: 0000 0000 0000 0000 0000 0000 0000 0000  ................

@@ -32,7 +32,7 @@
 Maximum mount count:      -1
 Last checked:             Fri May  3 16:14:49 2024
 Check interval:           0 (<none>)
-Lifetime writes:          1196 kB
+Lifetime writes:          73 kB
 Reserved blocks uid:      0 (user root)
 Reserved blocks gid:      0 (group root)
 First inode:              11
@@ -44,7 +44,7 @@
 Directory Hash Seed:      0b7f9cfd-0113-486c-a453-4f5483bd486b
 Journal backup:           inode blocks
 Checksum type:            crc32c
-Checksum:                 0x0fa98dfb
+Checksum:                 0xa5bbca64
 Checksum seed:            0xf81d767d
 Orphan file inode:        12
 Journal features:         (none)


The "Lifetime writes" being much higher on fat32 suggests that despite
"nodiscard", less zeroes were written out when ext4 or tmpfs are the underlying
FS?

Thanks!

cheers, josch
Download attachment "signature.asc" of type "application/pgp-signature" (834 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ