linux-ext4 - Re: How to fix up mballoc

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4A691BB7.60802@redhat.com>
Date:	Thu, 23 Jul 2009 21:25:59 -0500
From:	Eric Sandeen <sandeen@...hat.com>
To:	Theodore Tso <tytso@....edu>
CC:	Andreas Dilger <adilger@....com>, linux-ext4@...r.kernel.org
Subject: Re: How to fix up mballoc

Eric Sandeen wrote:
> Theodore Tso wrote:
>> On Thu, Jul 23, 2009 at 12:43:47PM -0500, Eric Sandeen wrote:
>>>> 1) In ext4_mb_normalize_request(), if the inode that we are allocating
>>>> does not have any open file descriptors for write (i.e., it's already
>>>> closed and we're allocating via delalloc) _and_ the inode was
>>>> previously opened with O_CREAT and without O_APPEND (checked via a
>>>> flag in EXT4_I(inode)), then do not normalize the size to a power of
>>>> two, but rather to the filesystem blocksize.
>>> I'm sort of woefully ignorant of a lot of the mballoc stuff.
>>>
>>> When you say once a file is written that's probably the final size... do
>>> you mean when writes are done and it's closed, or when the first write
>>> to the file is complete?
>>>
>>> I think an awful lot of normal cases write to a file in sub-file-sized
>>> chunks (think mp3 or flac encoding, file downloading, etc).
>> I meant when the writes are done and the files are closed; hence my
>> proposal that we do this do #1 above only if there are no open file
>> descriptors for write.  That is, if the file can be written and closed
>> by the userspace process before any delayed allocation blocks are
>> attempted to be written by the filesystem, we can probably safely
>> assume that the file won't grown in size later on.
> 
> Ah, ok.  Sorry, I misunderstood.  Yep, that seems reasonable.
> 
> It should probably get tested with workloads like video transcoding,
> where there will be incremental writes that span many minutes or hours.

Ugh right after I sent this I think I'm finally making sense of it.  :)
 In that case, come allocation time there =would= be file descriptors
open, and we'd go back to the old method of normalizing the allocation.
 You're just talking about changing things where an entire series of
file writes have come & gone, everything is closed & done, and -now-
we're allocating.

Sorry for being slow.  :)

-Eric
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html