linux-ext4 - Re: Significant difference in 'file size' and 'disk usage' for single files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Sun, 5 Nov 2017 12:16:12 +0100
From:   mfe555 <mfe555@....de>
To:     linux-ext4@...r.kernel.org
Cc:     mfe555 <mfe555@....de>
Subject: Re: Significant difference in 'file size' and 'disk usage' for single
 files

Some follow-up:

The issue only occurs with "bigalloc" enabled.

     echo 3 > /proc/sys/vm/drop_caches

seems to detach the blocked disk space from the files (so that 'du file' no longer includes the offset), but it does not free the space, 'df' still shows all file overheads as used disk space.



Am 02.11.2017 um 20:17 schrieb mfe555:
> Hi, I'm using ext4 on a Linux based Enigma2 set-top box, kernel 4.8.3.
>
> When creating a fresh file, there is a significant difference in file 
> size (ls -la) and disk usage (du). When making two copies of the file ..
>
> gbquad:/hdd/test# cp file file.copy1
> gbquad:/hdd/test# cp file file.copy2
> gbquad:/hdd/test# ls -la
> -rw-------    1 root     root     581821460 Nov  1 18:52 file
> -rw-------    1 root     root     581821460 Nov  1 18:56 file.copy1
> -rw-------    1 root     root     581821460 Nov  1 18:57 file.copy2
> gbquad:/hdd/test# du *
> 607232  file
> 658176  file.copy1
> 644864  file.copy2
>
> ... all three files show an overhead in the ~10% range, and the 
> overhead is different for these files although their md5sums are equal.
>
> When deleting a file (rm), the overhead remains occupied on the disk. 
> For example, after deleting "file", "df" reports approx. 581821460 
> more bytes free, not 607232 kbytes more free space. The overhead 
> (607232 kB - 581821460 B =pprox. 39 MB) remains blocked.
>
> When re-booting, the blocked space becomes free again, and in addition 
> the overhead of those files that were not deleted also disappears, so 
> that after a reboot the'file size' and 'disk usage' match for all 
> files (except for rounding up to some block size).
>
> A colleague and I have observed this on two different "kernel 4.8.3" 
> boxes and three ext4 disks, but not on a "kernel 3.14" box also using 
> ext4.
>
> Can anyone help me with this ?
>
> Thanks a lot
> Lukas
>