linux-ext4 - Re: Eric Whitney's ext4 scaling data

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130327033322.GB9887@gmail.com>
Date:	Wed, 27 Mar 2013 11:33:23 +0800
From:	Zheng Liu <gnehzuil.liu@...il.com>
To:	Theodore Ts'o <tytso@....edu>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: Eric Whitney's ext4 scaling data

On Tue, Mar 26, 2013 at 12:00:48AM -0400, Theodore Ts'o wrote:
> 
> Eric Whitney has very thoughtfully provided an updated set of ext4
> scalability data (with comparisons against ext3, xfs, and btrfs)
> comparing performance between 3.1 and 3.2, and comparing performance
> between 3.2 and 3.6-rc3.
> 
> I've made his compressed tar file available at:
> 
> https://www.kernel.org/pub/linux/kernel/people/tytso/ext4_scaling_data.tar.xz
> https://www.kernel.org/pub/linux/kernel/people/tytso/ext4_scaling_data.tar.gz
> 
> His comments on this data are:
> 
>    It contains two sets of data - one comparing 3.2 and 3.1 (this was
>    the last data set I posted publicly) and another comparing 3.6-rc3
>    and 3.2.  3.6-rc3 was the last data set I collected, and until now, I
>    hadn't prepared graphs for it.  The graphical results are consistent
>    with what I'd reported verbally over the first 2/3 of last year - not
>    much change between 3.2 and 3.6-rc3.  The last large change I could
>    see occurred in 3.2, as mentioned in the notes.
> 
>    The tarball unpacks into a directory named ext4_scaling_data and
>    contains a few subdirectories.  The directories named 3.2 and 3.6-rc3
>    map to the data sets described above.  Each contains a file named
>    index.html which you can open with a web browser to see the graphs,
>    browse the raw data, ffsb profiles and lockstats, etc.
> 
>    Hopefully you'll find the lockstats and other information useful,
>    even though stale (3.6-rc3 became available the last week in August
>    2012).
> 
> Thanks, Eric for making this data available!

Thanks for sharing this with us.  I have an rough idea that we can create
a project, which have some test cases to test the performance of file
system.  We can use fio to simulate all kinds of scenarios, such as

 - logger app (buffered io, append write, sequential write);
 - distribute file system (preallocate, buffered io, random read/write)
 - database (direct io, random read/write)
 - search (mmapped, random read, periodic append write)
 - ...

If we want to measure the performance of file system, we could simply
run a script to run some cases and get some result.

Currently we already have xfstests, but AFAIK it can verify that there
is no bug, deadlock, etc. in a file system, and it couldn't tell us
whether there has a performance regression after applied some patches.
(Please correct me if I am wrong.)  So the question is whether it is
worth creating a new project.  Or we should add these test cases into
xfstests.

Regards,
                                                - Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html