linux-ext4 - Re: transaction batching performance & multi-threaded synchronous writers

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Tue, 15 Jul 2008 10:05:32 -0400
From:	Josef Bacik <jbacik@...hat.com>
To:	Josef Bacik <jbacik@...hat.com>
Cc:	Andreas Dilger <adilger@....com>,
	Ric Wheeler <ricwheeler@...il.com>, linux-ext4@...r.kernel.org
Subject: Re: transaction batching performance & multi-threaded synchronous
	writers

On Tue, Jul 15, 2008 at 08:51:27AM -0400, Josef Bacik wrote:
> On Tue, Jul 15, 2008 at 01:58:32AM -0600, Andreas Dilger wrote:
> > On Jul 14, 2008  12:58 -0400, Josef Bacik wrote:
> > > Perhaps we track the average time a commit takes to occur, and then if
> > > the current transaction start time is < than the avg commit time we sleep
> > > and wait for more things to join the transaction, and then we commit.
> > > How does that idea sound?  Thanks,
> > 
> > The drawback of this approach is that if the thread waits an extra "average
> > transaction time" for the transaction to commit then this will increase the
> > average transaction time each time, and it still won't tell you if there
> > needs to be a wait at all.
> >
> 
> I'm not talking about the average transaction life, as you say it would be
> highly dependant on random things that have nothing to do with the transaction
> time (waiting for locks and such).  I'm measuring the time it takes for the
> actual commit to take place, so I record the start time in
> journal_commit_transaction when we set running_transaction = NULL, and then the
> end time right before the wakeup() at the end of journal_commit_transaction,
> that way there is an idea of how long the committing of a transaction to disk
> happens.  If we only have two threads doing work and fsyncing, its going to be a
> constant time, because we'll only be writing a certain number of buffers each
> time.
>  

Hmm now that I think about it you are right, if the thread waits then the time
the transaction was running could be greater than the commit time and this
wouldn't help.  Though one would hope that dirtying pagecache would still be
several times faster than writing to disk so any amount of waiting wouldn't be
too much of a big deal, and waiting too long would hurt in our current case
anyway.  Thanks,

Josef
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html