linux-ext4 - Re: transaction batching performance & multi-threaded synchronous writers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20080714165858.GA10268@unused.rdu.redhat.com>
Date:	Mon, 14 Jul 2008 12:58:58 -0400
From:	Josef Bacik <jbacik@...hat.com>
To:	Ric Wheeler <ricwheeler@...il.com>
Cc:	linux-ext4@...r.kernel.org
Subject: Re: transaction batching performance & multi-threaded synchronous
	writers

On Mon, Jul 14, 2008 at 12:15:23PM -0400, Ric Wheeler wrote:
>
> Here is a pointer to the older patch & some results:
>
> http://www.spinics.net/lists/linux-fsdevel/msg13121.html
>
> I will retry this on some updated kernels, but would not expect to see a 
> difference since the code has not been changed ;-)
>

I've been thinking, the problem with this for slower disks is that with the
patch I provided we're not really allowing multiple things to be batched, since
one thread will come up, do the sync and wait for the sync to finish.  In the
meantime the next thread will come up and do the log_wait_commit() in order to
let more threads join the transaction, but in the case of fs_mark with only 2
threads there won't be another one, since the original is waiting for the log to
commit.  So when the log finishes committing, thread 1 gets woken up to do its
thing, and thread 2 gets woken up as well, it does its commit and waits for it
to finish, and thread 2 comes in and gets stuck in log_wait_commit().  So this
essentially kills the optimization, which is why on faster disks this makes
everything go better, as the faster disks don't need the original optimization.

So this is what I was thinking about.  Perhaps we track the average time a
commit takes to occur, and then if the current transaction start time is < than
the avg commit time we sleep and wait for more things to join the transaction,
and then we commit.  How does that idea sound?  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html