linux-ext4 - Re: ext4

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 26 Jun 2012 09:06:54 -0400
From:	Eric Sandeen <sandeen@...hat.com>
To:	Fredrick <fjohnber@...o.com>
CC:	linux-ext4@...r.kernel.org
Subject: Re: ext4_fallocate

On 6/25/12 2:42 AM, Fredrick wrote:
> Hello Ext4 developers,
> 
> When calling fallocate on ext4 fs, ext4_fallocate does not initialize
> the extents. The extents are allocated only when they are actually
> written. This is causing a problem for us. Our programs create many
> "write only once" files as large as 1G on ext4 very rapidly at times.
> We thought fallocate would solve the problem. But it didnt.

What actual problem is this?  Fallocate does indeed allocate the extents -
it picks real, physical blocks on the disk, and allocates them to the file
in question.  If you map the file, you'll see those blocks.

The only thing it does not do is write data into them, but instead flags
them as unwritten, so that subsequent reads will return zeros.  So when
you say fallocate did not solve "this problem" which problem
do you mean?

> If I change the EXT4_GET_BLOCKS_CREATE_UNINIT_EXT to
> just EXT4_GET_BLOCKS_CREATE in the ext4_map_blocks in the ext4_fallocate call,
> the extents get created in fallocate call itself. This is helping us.
> Now the write throughtput to the disk was close to 98%. When extents were not
> initialized, our disk throughput were only 70%.
> 
> Can this change be made to ext4_fallocate?

Others addressed the serious problems with allowing fallocate to expose
bits of stale user data, but a couple things:

1) Can you describe the workload a bit more?  I'd be interested to know what
kind of workload you have which shows a 30% perf regression converting
unwritten extents.  So you fallocate 1G files, and then write to them how?

2) Are you sure zoho.com customers will be ok with the potential for
data exposure?  It's up to you I suppose to determine whether this is really
a risk, but you need to fully understand the implications of the patches
others have provided.  There are good reasons I and others have resisted
merging them upstream.

-Eric

> Thanks,
> Fredrick
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html