linux-kernel - Re: xfs: does mkfs.xfs require fancy switches to get decent performance? (was Tux3 Report: How fast can we fsync?)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 11 May 2015 17:12:39 -0700 (PDT)
From:	David Lang <david@...g.hm>
To:	Daniel Phillips <daniel@...nq.net>
cc:	Pavel Machek <pavel@....cz>, Howard Chu <hyc@...as.com>,
	Mike Galbraith <umgwanakikbuti@...il.com>,
	Dave Chinner <david@...morbit.com>,
	linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org,
	tux3@...3.org, "Theodore Ts'o" <tytso@....edu>,
	OGAWA Hirofumi <hirofumi@...l.parknet.co.jp>
Subject: Re: xfs: does mkfs.xfs require fancy switches to get decent performance?
 (was Tux3 Report: How fast can we fsync?)

On Mon, 11 May 2015, Daniel Phillips wrote:

> On 05/11/2015 03:12 PM, Pavel Machek wrote:
>>>> It is a fact of life that when you change one aspect of an intimately interconnected system,
>>>> something else will change as well. You have naive/nonexistent free space management now; when you
>>>> design something workable there it is going to impact everything else you've already done. It's an
>>>> easy bet that the impact will be negative, the only question is to what degree.
>>>
>>> You might lose that bet. For example, suppose we do strictly linear allocation
>>> each delta, and just leave nice big gaps between the deltas for future
>>> expansion. Clearly, we run at similar or identical speed to the current naive
>>> strategy until we must start filling in the gaps, and at that point our layout
>>> is not any worse than XFS, which started bad and stayed that way.
>>
>> Umm, are you sure. If "some areas of disk are faster than others" is
>> still true on todays harddrives, the gaps will decrease the
>> performance (as you'll "use up" the fast areas more quickly).
>
> That's why I hedged my claim with "similar or identical". The
> difference in media speed seems to be a relatively small effect
> compared to extra seeks. It seems that XFS puts big spaces between
> new directories, and suffers a lot of extra seeks because of it.
> I propose to batch new directories together initially, then change
> the allocation goal to a new, relatively empty area if a big batch
> of files lands on a directory in a crowded region. The "big" gaps
> would be on the order of delta size, so not really very big.

This is an interesting idea, but what happens if the files don't arrive as a big 
batch, but rather trickle in over time (think a logserver that if putting files 
into a bunch of directories at a fairly modest rate per directory)

And when you then decide that you have to move the directory/file info, doesn't 
that create a potentially large amount of unexpected IO that could end up 
interfering with what the user is trying to do?

David Lang
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/