linux-ext4 - Re: ext4 fallocate+truncate not freeing up allocated space

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <AANLkTikygDP5HoZRNy5yUga-BPxNTZwKVd6chW1O-DUA@mail.gmail.com>
Date:	Fri, 29 Oct 2010 12:10:32 -0700
From:	Curt Wohlgemuth <curtw@...gle.com>
To:	George Kola <georgekola@...il.com>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: ext4 fallocate+truncate not freeing up allocated space

Are you using FALLOC_FL_KEEP_SIZE?  If so, commit
a379cd1d6bb00f9f5d2759d4a5621a884df5914e fixed a bug with this (but I
don't know offhand which kernel this went into).

Curt

On Fri, Oct 29, 2010 at 11:26 AM, George Kola <georgekola@...il.com> wrote:
> I am using RHEL (CentOS) 5.5 with kernel  2.6.18-194.17.4.el5  and ext4. I
> posted to RHEL bugzilla and they requested me to post it here.
>
> I wrote a program that fallocates space for a file. It does not know the
> file
> size apriori and uses a heuristic. If the final file size is less than the
> allocated space, I free up the extra space with truncate. I found that
> freeing
> up did not work so I ended up doing two truncates -- truncate(size+1)
> followed
> by truncate(size) as suggested in
> http://kerneltrap.org/mailarchive/linux-ext4/2009/7/21/6243463.
>
> I now find that the freed space is not given back till a unmount+ mount.
> I detected this by finding disks full even though the used space was lesser.
>
> I did the standard tests
> 1. Stopped all program having handles open to files on that partition
> 2. Verified with lsof that there are no open handles to files in that
> partition
>      Still there was a mismatch. e.g. see below
>
>
> e.g.
> bash-3.2# mount | grep hadoop_data
> /dev/mapper/vg0-data on /hadoop_data type ext4
> (rw,noatime,barrier=0,journal_checksum)
> bash-3.2# mount | grep hadoop_data
> /dev/mapper/vg0-data on /hadoop_data type ext4
> (rw,noatime,barrier=0,journal_checksum)
> bash-3.2# du -h /hadoop_data/ | tail -1
> 328G    /hadoop_data/
> bash-3.2# df -h /hadoop_data/
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/mapper/vg0-data  745G  592G  117G  84% /hadoop_data
>
>          After unmount + mount, I find this
>
> bash-3.2# umount /hadoop_data/
> bash-3.2# mount /hadoop_data/
> bash-3.2# df -h /hadoop_data/
> Filesystem            Size  Used Avail Use% Mounted on
> /dev/mapper/vg0-data  745G  328G  380G  47% /hadoop_data
>
>
>
>
>
> Version-Release number of selected component (if applicable):
>
> RHEL (CentOS) 5.5 with kernel   2.6.18-194.17.4.el5  and ext4.
>
>
> How reproducible:
>  All the time.
>
>
>
> Steps to Reproduce:
>   Have a program
> 1.  Open a file on ext4 partition
> 2. Fallocate space for that file  say 20 GB
> 3. Write data to the file  (but less than the allocated space) say 10 GB
> 4. Close the file
> 5. Call truncate(file-path, size+1), truncate(file-path,size) on that file
> to
> give up the space (observed that the space is given up using filefrag)
> 6. Use du and df on that ext4 partition to find that the mismatch
>
>
> -George
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html