linux-ext4 - [PATCH] fix strange values displayed by e2freefrag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <32C2860D-6E4D-41CB-8C80-29BAF1C153D4@dilger.ca>
Date:	Thu, 29 Jul 2010 18:03:02 -0600
From:	Andreas Dilger <adilger@...ger.ca>
To:	horhe <skrzynka_na_spami@...ekcja.mejor.pl>
Cc:	linux-ext4@...r.kernel.org
Subject: [PATCH] fix strange values displayed by e2freefrag

On 2010-07-24, at 15:10, horhe wrote:
> Today i run e2freefrag on ext4 partition and i saw:
> Device: /dev/mapper/vg--bez--raidu-lvarchiwumnowe
> Blocksize:           4096 bytes
> Total blocks:    52166656
> Free blocks:        57254 (0.1%)
Here is the problem  ^^^^^^^^^^^^^^^

> Min. free extent: 4 KB
> Max. free extent: 128120 KB
> Avg. free extent: 3396 KB
> 
> HISTOGRAM OF FREE EXTENT SIZES:
> Extent Size Range :  Free extents   Free Blocks  Percent
>   4K...    8K-  :          1500          1500    2.62%
>   8K...   16K-  :          1566          3933    6.87%
>  16K...   32K-  :          1112          5707    9.97%
>  32K...   64K-  :           228          2480    4.33%
>  64K...  128K-  :           222          5061    8.84%
> 128K...  256K-  :           269         12364   21.59%
> 256K...  512K-  :           307         28279   49.39%
> 512K... 1024K-  :           487         84412  147.43%
>   1M...    2M-  :           621        240621  420.27%
>   2M...    4M-  :           780        574302  1003.08%
>   4M...    8M-  :           803       1162903  2031.13%
>   8M...   16M-  :           488       1317526  2301.19%
>  16M...   32M-  :           312       1728598  3019.17%
>  32M...   64M-  :           119       1269059  2216.54%
>  64M...  128M-  :            49       1088111  1900.50%
> 
> And I'm suprised when I can see so high percents, I don't know how to
> interpret this values.
> 
> I was completly full partition, then i freed 15% space (about 30G), then I saw strange values.  I can't reproduce this problem, i filled up partition, removed files and problem doesn't appear.

That's because the kernel does not update the "free blocks" summary value in the superblock on disk.  It only updates the per-group free blocks counters in the group descriptor table.  That means the "free blocks" value read from the filesystem is totally incorrect (too low in your case) and caused the percentage to be incorrect.

For a while, when calling statfs() to get the free blocks information, it would also write out the superblock summary values.  However, this confused the journal checksum code because the modification wasn't being checksummed correctly.

It is possible to change the e2freefrag code to count the number of free blocks it finds, instead of depending on the value in the superblock (which may be slightly out of date no matter how often it is updated).  A patch to fix this, and to make e2freefrag work properly on filesystems with more than 2^32 free blocks, is attached.

I'd still like some way for userspace to update the superblock, or have the kernel do it periodically, so that "dumpe2fs -h" and other tools that look at the superblock are at least close instead of having values from the time of last mount.

Cheers, Andreas

Download attachment "e2freefrag-blocks.diff" of type "application/octet-stream" (8330 bytes)