linux-ext4 - Re: 2.6.33.1: ext4 disk free weirdness

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4BB0E884.5070500@redhat.com>
Date:	Mon, 29 Mar 2010 12:51:00 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Justin Maggard <jmaggard10@...il.com>
CC:	ext4 development <linux-ext4@...r.kernel.org>
Subject: Re: 2.6.33.1: ext4 disk free weirdness

Justin Maggard wrote:
> On Mon, Mar 29, 2010 at 8:01 AM, Eric Sandeen <sandeen@...hat.com> wrote:
>> tytso@....edu wrote:
>>> On Mon, Mar 29, 2010 at 09:48:50AM -0500, Eric Sandeen wrote:
>>>> CaT wrote:
>>>>> Using kernel 2.6.33.1. Am rsyncing from /data/mirror (a raid0 mount) to
>>>>> /dev/sda1 (a usb HD). Both filesystems formatted as ext3 but mounted as
>>>>> ext4. /mnt filesystem is only being added to and currently large files
>>>>> (iso files 700MB-4.4GB) are being copied. When I remounted /mnt as ext3
>>>>> I could not duplicate the issue and the amount of disk used stabilised
>>>>> at 182G. The df results, whilst /mnt is mounted as ext4, are below:
>>>> I would say this is likely speculative allocation due to delalloc
>>>> (ext4 has to reserve worst-case metadata amounts to prepare for
>>>> delalloc writeback).  When the file data actually gets written out,
>>>> the worst-case reservation is freed up again.  We also flush and
>>>> switch to nodelalloc when the filesystem is pretty close to full.
>>>> see ext4_nonda_switch() for example.
>>>>
>>>> This looks pretty severe though, more than I would expect
>>>> from that behavior.
>>> This is fixed in commit d330a5bef, which which got merged post
>>> 2.6.34-rc2, and which we need to get into 2.6.33 stable.  If you
>>> cherry-pick that commit, I think you'll see that it fixes this
>>> problem.
>> Oh, right, spaced that one.  Thanks for the reminder Ted.  :)
>>
>> And from now on ext3-mounted-as-ext4 is special-cased so you won't get
>> delalloc, and you won't get this behavior at all (nor will you get
>> any delalloc benefits, FWIW).
>>
> 
> Out of curiosity, when you say ext3-mounted-as-ext4, are you referring
> to filesystems without ext4 features mounted -t ext4, or filesystems
> mounted -t ext3 when CONFIG_EXT4_USE_FOR_EXT23 is defined?

hmmm to be honest i've lost track a little ;)

I was referring to:

commit ba69f9ab7df844125898104e854e063b47c26637
Author: Jan Kara <jack@...e.cz>
Date:   Wed Mar 24 20:18:37 2010 -0400

    ext4: Don't use delayed allocation by default when used instead of ext3
    
    When ext4 driver is used to mount a filesystem instead of the ext3 file
    system driver (through CONFIG_EXT4_USE_FOR_EXT23), do not enable delayed
    allocation by default since some ext3 users and application writers have
    developed unfortunate expectations about the safety of writing files on
    systems subject to sudden and violent death without using fsync().
    
    Signed-off-by: Jan Kara <jack@...e.cz>
    Signed-off-by: "Theodore Ts'o" <tytso@....edu>

looking at that commit I guess it's -only- when CONFIG_EXT4_USE_FOR_EXT23
is defined.  So in answer to your question above, it's your latter case.

See previous complaint about this all becoming rather confusing....

-Eric


> -Justin

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html