linux-kernel - [performance bug] kernel building regression on 64 LCPUs machine

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 19 Jan 2011 09:55:48 +0800
From:	"Alex,Shi" <alex.shi@...el.com>
To:	czoccolo@...il.com
Cc:	vgoyal@...hat.com, jaxboe@...ionio.com,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"Li, Shaohua" <shaohua.li@...el.com>,
	"Chen, Tim C" <tim.c.chen@...el.com>
Subject: [performance bug] kernel building regression on 64 LCPUs machine

Shaohua and I tested kernel building performance on latest kernel. and
found it is drop about 15% on our 64 LCPUs NHM-EX machine on ext4 file
system. We find this performance dropping is due to commit
749ef9f8423054e326f. If we revert this patch or just change the
WRITE_SYNC back to WRITE in jbd2/commit.c file. the performance can be
recovered. 

iostat report show with the commit, read request merge number increased
and write request merge dropped. The total request size increased and
queue length dropped. So we tested another patch: only change WRITE_SYNC
to WRITE_SYNC_PLUG in jbd2/commit.c, but nothing effected.

we didn't test deadline IO mode, just test cfq. seems insert write
request into sync queue effect much read performance, but we don't know
details. What's your comments of this? 

iostat of .37 kernel:
rrqm/s   wrqm/s r/s     w/s     rMB/s  wMB/s avgrq-sz avgqu-sz   await  svctm  %util
22.5772 96.46 92.3742 14.747  1.0048  0.439474 34.8557 0.18078 3.8076 0.30447 2.94302
iostat of commit reverted .37:
26.6223 80.21 107.875 6.03538 1.51415 0 41.3275 0.153385 3.80569 0.377231 3.22323

vmstat report show, read bandwidth dropping: 
vmstat of .37:
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
3.4 52.6 0.0 64303937.0 16466.7 121544.5 0.0 0.0 2102.7 1914.6 7414.1 3185.7 2.0 1.0 80.3 16.7 0.0
vmstat of revert all from .37
2.2 35.8 0.0 64306767.4 17265.6 126101.2 0.0 0.0 2415.8 1619.1 8532.2 3556.2 2.5 1.1 83.0 13.3 0.0

Regards
Alex

===
diff --git a/fs/jbd/commit.c b/fs/jbd/commit.c
index 34a4861..27ac2f3 100644
--- a/fs/jbd/commit.c
+++ b/fs/jbd/commit.c
@@ -294,7 +294,7 @@ void journal_commit_transaction(journal_t *journal)
 	int first_tag = 0;
 	int tag_flag;
 	int i;
-	int write_op = WRITE_SYNC;
+	int write_op = WRITE;
 
 	/*
 	 * First job: lock down the current transaction and wait for
diff --git a/fs/jbd2/commit.c b/fs/jbd2/commit.c
index f3ad159..69ff08e 100644
--- a/fs/jbd2/commit.c
+++ b/fs/jbd2/commit.c
@@ -329,7 +329,7 @@ void jbd2_journal_commit_transaction(journal_t *journal)
 	int tag_bytes = journal_tag_bytes(journal);
 	struct buffer_head *cbh = NULL; /* For transactional checksums */
 	__u32 crc32_sum = ~0;
-	int write_op = WRITE_SYNC;
+	int write_op = WRITE;
 
 	/*
 	 * First job: lock down the current transaction and wait for


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/