linux-ext4 - Re: Bug in extent zeroout: blocks not marked as new

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1259011043.25937.29.camel@bobble.smo.corp.google.com>
Date:	Mon, 23 Nov 2009 13:17:23 -0800
From:	Frank Mayhar <fmayhar@...gle.com>
To:	tytso@....edu
Cc:	Curt Wohlgemuth <curtw@...gle.com>,
	ext4 development <linux-ext4@...r.kernel.org>
Subject: Re: Bug in extent zeroout: blocks not marked as new

On Mon, 2009-11-23 at 14:50 -0500, tytso@....edu wrote:
> However, that doesn't explaining the corruption which you are seeing,
> which is zero blocks showing up where they shouldn't --- since the
> freed dirty metadata blocks that we would be zapping with
> unmap_underlying_metadata() are highly unlikely to be all zero's.
> 
> Am I missing something?

I think the missing bit of information here is that this is happening on
converted ext2 partitions.  Older non-extent-based files are being
gradually removed while new extent-based files are being created.  Last
week, Curt found that in certain cases newly-reallocated metadata blocks
are being hit; these metadata blocks just happen to be newly-emptied
indirect blocks (freed by a recent delete), which just happen to have
been filled with zeroes as they were emptied.  As I recall, Curt found
that these were pretty clearly the source of the blocks being written in
this case.

To elaborate (per Curt), we're seeing these dirty metadata writes from
freed indirect-block inodes because ext4_free_branches() calls
ext4_handle_dirty_metadata() on an indirect block after freeing (some or
all of) its data blocks.  We do the same thing in ext3_free_branches()
but ext2_free_branches() DOES NOT mark the metadata block as dirty, so
there will be a lot more dirty metadata floating about in ext[34] than
in ext2 and hence a lot more chance for the race we've been seeing.

Finally, we have a question about the zero-out path:  Is there any
known, concrete improvement given by doing the zero-out as opposed to
just continuing to split the extents?  At the moment, by the way, there
is one definite problem: Since it doesn't try to do a merge left (which
it should) it invariably leaves a 14-block extent fragment, thus
increasing fragmentation of the file.  It's not a huge problem (since
the extents are in fact contiguous) but it's there.
-- 
Frank Mayhar <fmayhar@...gle.com>
Google, Inc.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html