| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Fri, 3 Jan 2014 23:54:21 +0800
From: fengguang.wu@...el.com
To: Josef Bacik <jbacik@...ionio.com>
Cc: linux-btrfs <linux-btrfs@...r.kernel.org>,
LKML <linux-kernel@...r.kernel.org>, lkp@...ux.intel.com
Subject: 1be41b78: +18% increased btrfs write throughput
Hi Josef,
FYI. We are doing 0day performance tests and happen to notice that
btrfs write throughput increased considerably during v3.10-11 time
frame:
v3.10 v3.11 v3.12 v3.13-rc6
--------------- ------------------------- ------------------------- -------------------------
50619 ~ 1% +17.0% 59209 ~ 2% +18.8% 60159 ~ 2% +20.5% 61007 ~ 0% lkp-ws02/micro/dd-write/11HDD-JBOD-cfq-btrfs-1dd
50619 +17.0% 59209 +18.8% 60159 +20.5% 61007 TOTAL iostat.sdd.wkB/s
and it's contributed by
commit 1be41b78bc688fc634bf30965d2be692c99fd11d
Author: Josef Bacik <jbacik@...ionio.com>
AuthorDate: Wed Jun 12 13:56:06 2013 -0400
Commit: Josef Bacik <jbacik@...ionio.com>
CommitDate: Mon Jul 1 08:52:28 2013 -0400
Btrfs: fix transaction throttling for delayed refs
Dave has this fs_mark script that can make btrfs abort with sufficient amount of
ram. This is because with more ram we can keep more dirty metadata in cache
which in a round about way makes for many more pending delayed refs. What
happens is we end up not throttling the transaction enough so when we go to
commit the transaction when we've completely filled the file system we'll
abort() because we use all of the space in the global reserve and we still have
delayed refs to run. To fix this we need to make the delayed ref flushing and
the transaction throttling dependant upon the number of delayed refs that we
have instead of how much reserved space is left in the global reserve. With
this patch we not only stop aborting transactions but we also get a smoother run
speed with fs_mark and it makes us about 10% faster. Thanks,
Reported-by: David Sterba <dsterba@...e.cz>
Signed-off-by: Josef Bacik <jbacik@...ionio.com>
fs/btrfs/ctree.h | 2 ++
fs/btrfs/extent-tree.c | 61 ++++++++++++++++++++++++++++++++++++++++++++++----
fs/btrfs/transaction.c | 24 +++++++++-----------
3 files changed, 69 insertions(+), 18 deletions(-)
time.elapsed_time
740 ++------------------*------------------------------------------------+
*.. .*.*..* *..*.*..*. |
720 ++ *.*..*.*. *..*.*..*. .*..*. .*..*. .*. |
| *. *..* *. *..*
700 ++ |
| |
680 ++ |
| |
660 ++ |
| |
640 ++ |
| |
620 ++ O O |
O O O O O O O O O O O O O O O O O O O O O O O O |
600 ++-------------------------------------------------------------------+
time.voluntary_context_switches
100000 ++------------------*-*----*--------------------------------------+
90000 ++ *. .*..*.*. *. *.. |
*. .. *..* .*.. .*..*.*.*..*.*..*.*..* .*
80000 ++* * * + .* |
70000 ++ *. |
| |
60000 ++ |
50000 ++ |
40000 ++ |
| |
30000 ++ |
20000 ++ |
| |
10000 ++ |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
time.file_system_inputs
800000 ++--------------------*----*--------------------------------------+
| *. .*..*.*..* *. *.. |
700000 *+ .. *..* .*.. .*..*.*.*..*.*..*.*..* .*
| * * * + .* |
600000 ++ *. |
500000 ++ |
| |
400000 ++ |
| |
300000 ++ |
200000 ++ |
| |
100000 ++ |
| |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
time.file_system_outputs
900000 ++----------------------------------------------------------------+
| *.*.. .*.*. .*. .*.*.*..*. .* |
800000 *+ .. *.*..*.*. *. *.. .*..*.*. *..*.*. + .*.*
700000 ++* * *. |
| |
600000 ++ |
500000 ++ |
| |
400000 ++ |
300000 ++ |
| |
200000 ++ |
100000 ++ |
| |
0 O+O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O--O-O-O--O-O--O-O--O-O----+
perf-stat.page-faults
6.6e+06 ++---------------------------------------------------------------+
| |
6.4e+06 ++ .*.*.. .*.. |
*. .*.*..*.*.*. *.* *. .*. .*..*. |
6.2e+06 ++*. *. *..*.* *..*.*.*..*.*..*.*
| |
6e+06 ++ |
| |
5.8e+06 ++ |
| |
5.6e+06 ++ |
| |
5.4e+06 ++O O |
O O O O O O O O O O O O O O O O O O O O O O O O |
5.2e+06 ++---------------------------------------------------------------+
perf-stat.minor-faults
6.6e+06 ++---------------------------------------------------------------+
| |
6.4e+06 ++ .*.*.. .*.. |
*. .*.*..*.*.*. *.* *. .*. .*..*. |
6.2e+06 ++*. *. *..*.* *..*.*.*..*.*..*.*
| |
6e+06 ++ |
| |
5.8e+06 ++ |
| |
5.6e+06 ++ |
| |
5.4e+06 ++O O |
O O O O O O O O O O O O O O O O O O O O O O O O |
5.2e+06 ++---------------------------------------------------------------+
vmstat.io.bi
600 ++-----------*-*--*-*--*--*----*-------------------------------------+
| .. *. + |
500 *+.*.*..*.* *..*.*..*.*.. .*.*..*.*..*.*..*.*..*
| *. |
| |
400 ++ |
| |
300 ++ |
| |
200 ++ |
| O O |
| |
100 ++ O |
O O O O O O O O O O O O O O O O O O O O O O |
0 ++-----------------------------------------------O-------------------+
vmstat.io.bo
660000 ++----------------------------------------------------------------+
| O O O O O O O O O |
640000 O+ O O O O O O O O O O O O O |
| O O O |
620000 ++ |
| |
600000 ++ |
| |
580000 ++ |
| |
560000 ++ .*.|
| .*. .*.*..*. .*..*.*..*.*..*.*. *
540000 *+*..*.*..*. .*.*.. .*. *. * |
| *..*.*..* * |
520000 ++----------------------------------------------------------------+
iostat.sde.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 ++ O O O O O |
680 O+ O O O O O O O O O O O |
| O O O O O O O |
660 ++ |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++ .*.*..*.*.. .*.*.. .*.. .*. *. .*..|
*..*.*..*.*. *. *.*.. .*..* *. *..*. .. *..* *
560 ++----------------------------------*-------------------*------------+
iostat.sde.avgqu-sz
125 ++-------------------------------------------------------------------+
| O |
120 ++ O O |
O O O O O O O O O O O O O O O O |
| O O O O O O O |
115 ++ |
| |
110 ++ |
| |
105 ++ |
| .*.. |
| .*. .* *..*.*..* .*. .*. .*.*.. .*..*
100 *+ *.. .*. + .*..*.*.. + *. *. *. * |
| * *..*.*. * |
95 ++-------------------------------------------------------------------+
iostat.sdf.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 ++ O O O O O O |
680 O+ O O O O O O O O |
| O O O O O O O O O |
660 ++ |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| .* |
580 ++ *.*.. .*..*. + .*.. .*..*.*.. *..|
|.. .*..*.*..* *..*.*.. .*..* *..*.*..* + *
560 *+------*---------------------------*---------------------------*----+
iostat.sdg.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O |
680 ++ O O O O O O O O O O O O O |
| O O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| .*.. .*.. |
580 ++ .*..* *.*..*. .*..*.*..*. .*.*..*..*.*..*.*..* .*..*
560 ++.* *. *..*.*. * |
*. |
540 ++-------------------------------------------------------------------+
iostat.sdd.avgrq-sz
720 ++-------------------------------------------------------------------+
| O |
700 O+ O O O O O O O O |
680 ++ O O O O O O O O O O O O |
| O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++ .*..*.*..*.*.. .*..*..*.*.. .*.. .*.*..*.*.. .*.. .*..*
560 *+.* * *.*.. .*..* *. * * |
| * |
540 ++-------------------------------------------------------------------+
iostat.sdd.avgqu-sz
120 ++-------------------------------------------------------------------+
| O O O O O O O O O O O O |
115 O+ O O O O O O O O O O O O O |
| |
| |
110 ++ |
| |
105 ++ |
| |
100 ++ *. *.. .*. .*.*..*. .*.*..|
|.. *..*.*..*.*..*. .*..*. .*.. + *. *. *. *
* *. *..* .*..* |
95 ++ * |
| |
90 ++-------------------------------------------------------------------+
iostat.sdh.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O O O O |
680 ++ O O O O O O O O O O O O |
| O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++.*.*..*.*.. .*..*.*.. .*.*..*. .*.*.. .*. .*. .*.*.. .*..*
560 *+ * *. *..*.*. *. *. *. * |
| |
540 ++-------------------------------------------------------------------+
iostat.sdh.avgqu-sz
120 ++-------------------------------------------------------------------+
| O O O O O O O O |
| O O O O O O O O O O |
115 O+ O O O O O O |
| O |
| |
110 ++ |
| |
105 ++ |
| |
| |
100 ++.*. *.. .*.. *. .*. .*..|
*. *..*. .*.. .*.. + * .. *. *..*.*..* *
| *..* *.*..*..* .*..* * |
95 ++-----------------------------*-------------------------------------+
iostat.sdi.avgrq-sz
720 ++-------------------------------------------------------------------+
| O O |
700 O+ O O O O O O O O |
680 ++ O O O O O O O O O O |
| O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| .*. .*.. .*..|
580 ++ .*.*..*.*..*. .*. *..*. *.. .*..*..*.*..*. .* * |
560 *+.*.*. *. *.. + * *. *
| * |
540 ++-------------------------------------------------------------------+
iostat.sdi.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O |
| O O O O O O O |
O O O O O O O O O O O |
115 ++ O O O O O O |
| O |
110 ++ |
| |
105 ++ |
| |
| .*.. .*..|
100 ++ *.*..*.*..*. .*.. .*..*.*..*..*.*..* *. .* |
|.. *..*.*. *.*..*.*..* *. *
95 *+-------------------------------------------------------------------+
iostat.sdj.avgqu-sz
120 ++---------------------------------------------------------O---------+
| O O O O O O O |
115 O+ O O O O O O O O O O O O O |
| O O O O |
| |
110 ++ |
| |
105 ++ |
| * |
100 ++.*.*..*. *.. + + .*.. *..|
*. *..*. + *.*.. + *..*.*..* + *
| *..*. .*..*.*..*.*..* * * |
95 ++ *. |
| |
90 ++-------------------------------------------------------------------+
iostat.sdk.avgrq-sz
720 ++-------------------------------------------------------------------+
| |
700 ++ O O O O O O O O O |
680 O+ O O O O O O |
| O O O O O O O O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
600 ++ |
| |
580 ++.*. .*.. .*.. *.. *. .*.. .*. .*. .*.. .*..*
*. *..* * *. .. *. .. *.. .*..* *. *. *..* * |
560 ++------------------*-------*-------*--------------------------------+
iostat.sdk.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O |
O O O O O O O O O O O O |
115 ++ O O O O O O O O O O O O O |
| |
110 ++ |
| |
105 ++ |
| .*.. |
100 ++.*. .*. .*.. .*..* .*.*..*. .*
*. *. *..*. .*..*. .* * *. *..*.*..*.*. |
95 ++ *..*.*. *. |
| |
90 ++-------------------------------------------------------------------+
iostat.sdl.avgrq-sz
720 ++-------------------------------------------------------------------+
O |
700 ++ O O O O O O |
680 ++ O O O O O O O O O O O O O O |
| O O O O O |
660 ++ |
640 ++ |
| |
620 ++ |
600 ++ |
| *..|
580 ++ .*..*.*..*.*..* *..*.*..*. .*.*.. .*. .*. .*.*.. + *
560 *+.* + .. *..*.*. *. *. *. * |
| * |
540 ++-------------------------------------------------------------------+
iostat.sdl.avgqu-sz
125 ++-------------------------------------------------------------------+
| |
120 ++ O O |
| O O O O O O O O |
O O O O O O O O O O O O O |
115 ++ O O O |
| |
110 ++ |
| |
105 ++ |
| .*. |
| .*..*. .*.*.. .*. .*.*. *.. .*..|
100 ++.* *..* .*..* *.*.. .*. *. *. * *
*. + .*.*. + .. * |
95 ++-------------*------------*----------------------------------------+
iostat.sdm.avgrq-sz
700 ++---------------------O-------O-------------------------------------+
O O O O O O O O O O O O |
680 ++ O O O O O O O O O |
| O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
| |
600 ++ |
| .*.. *..|
580 ++ *.. .*..* .*.. .*.*..*. .*.*.. .*. .*. .*.*.. + *
| + * * *. *.. .*. *. *. *. * |
560 *+-*--------------------------------*--------------------------------+
iostat.sdn.avgrq-sz
700 ++------------------O----------O-------------------------------------+
O O O O O O O O O |
680 ++ O O O O O O O O O O O O O O |
660 ++ O |
| |
640 ++ |
| |
620 ++ |
| |
600 ++ |
580 ++ .*..* .*..|
*..*. .*.*.. .*.*. + .*. .*.. .*.. .* *. .*..* *
560 ++ *. *.*. *. *..* * *. + .. *..* |
| * |
540 ++-------------------------------------------------------------------+
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists