linux-ext4 - Re: [PATCH v3 0/6] ext4: extents status tree shrinker improvement

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20141107023840.GA5420@gmail.com>
Date:	Fri, 7 Nov 2014 10:38:40 +0800
From:	Zheng Liu <gnehzuil.liu@...il.com>
To:	Jan Kara <jack@...e.cz>
Cc:	刘峥(文卿) <wenqing.lz@...baba-inc.com>,
	Theodore Ts'o <tytso@....edu>, linux-ext4@...r.kernel.org,
	刘峥(文卿) <wenqing.lz@...bao.com>,
	Andreas Dilger <adilger.kernel@...ger.ca>
Subject: Re: [PATCH v3 0/6] ext4: extents status tree shrinker improvement

On Mon, Nov 03, 2014 at 05:10:46PM +0100, Jan Kara wrote:
>   Hello,
> 
> On Tue 21-10-14 23:58:10, 刘峥(文卿) wrote:
> > I deeply sorry for this because of my delay work.  I don’t have any objection
> > for Jan’s suggestions.  Until now there are still some works that push me
> > tough, and I can see that I don’t have time to finish it at this merge
> > window.  It’s a shame for me!
> > 
> > Jan, I really really appreciate if you are willing to push this patch set
> > to completion.  Thanks!!!
>   OK, I have updated the patches according to the review I and Ted did. It
> survives basic fsstress run. How were you testing your patches? I should
> probably also gather some statistics etc...

Thanks!!

Here are my test cases for performance.

case 1:
  [global]
  ioengine=psync
  bs=4k
  directory=/mnt/sda1
  thread
  group_reporting
  fallocate=0
  direct=0
  filesize=10g
  size=20g
  runtime=300
  
  [io]
  rw=randwrite:32
  rw_sequencer=sequential
  numjobs=25
  nrfiles=10

case 2:
  [global]
  ioengine=psync
  bs=4k
  directory=/mnt/sda1
  group_reporting
  fallocate=0
  direct=0
  filesize=10g
  size=20g
  runtime=300
  
  [io]
  rw=write:4k
  numjobs=15
  nrfiles=20000

For getting a really fragmented extent status tree, I will disable
extent status tree merge as I run these test cases.  The patch looks
like below:

diff --git a/fs/ext4/extents_status.c b/fs/ext4/extents_status.c
index 09fd576..0946f50 100644
--- a/fs/ext4/extents_status.c
+++ b/fs/ext4/extents_status.c
@@ -351,6 +351,7 @@ static void ext4_es_free_extent(struct inode *inode, struct extent_status *es)
 static int ext4_es_can_be_merged(struct extent_status *es1,
 				 struct extent_status *es2)
 {
+#if 0
 	if (ext4_es_status(es1) != ext4_es_status(es2))
 		return 0;
 
@@ -376,6 +377,7 @@ static int ext4_es_can_be_merged(struct extent_status *es1,
 	/* we need to check delayed extent is without unwritten status */
 	if (ext4_es_is_delayed(es1) && !ext4_es_is_unwritten(es1))
 		return 1;
+#endif
 
 	return 0;
 }


In the mean time, the following sysctl parameters are adjusted to keep
dirty data in memory as much as possible.

sudo sysctl vm.dirty_ratio=80
sudo sysctl vm.dirty_background_ratio=60

Thanks,
                                                - Zheng
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html