linux-ext4 - Re: bigalloc and max file size

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <7EB4969C-BE2B-49A6-9905-7694D14C7A5F@whamcloud.com>
Date:	Mon, 31 Oct 2011 13:09:34 -0600
From:	Andreas Dilger <adilger@...mcloud.com>
To:	Sunil Mushran <sunil.mushran@...cle.com>
Cc:	Tao Ma <tm@....ma>, Theodore Tso <tytso@....EDU>,
	linux-ext4 development <linux-ext4@...r.kernel.org>,
	Alex Zhuravlev <bzzz@...mcloud.com>,
	"hao.bigrat@...il.com" <hao.bigrat@...il.com>
Subject: Re: bigalloc and max file size

On 2011-10-31, at 12:53 PM, Sunil Mushran wrote:
> On 10/31/2011 03:27 AM, Tao Ma wrote:
>> OK, so let me explain why the big cluster length works.
>> 
>> In the new bigalloc case if chunk size=64k, and with the linux-3.0
>> source, every file will be allocated a chunk, but they aren't contiguous
>> if we only write the 1st 4k bytes. In this case, writeback and the block
>> layer below can't merge all the requests sent by ext4. And in our test
>> case, the total io will be around 20000. While with the cluster size, we
>> have to zero the whole cluster. From the upper point of view. we have to
>> write more bytes. But from the block layer, the write is contiguous and
>> it can merge them to be a big one. In our test, it will only do around
>> 2000 ios. So it helps the test case.
> 
> Am I missing something but you cannot zero the entire cluster because
> block_write_full_page() drops pages past i_size.
> 
> http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commitdiff;h=5693486bad2bc2ac585a2c24f7e2f3964b478df9

With ext4_ext_zeroout->blkdev_issue_zeroout() it submits the zeroing
request directly to the block layer (with cloned ZERO_PAGE pages) and
skips the VM entirely.

Cheers, Andreas
--
Andreas Dilger 
Principal Engineer
Whamcloud, Inc.



--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html