linux-kernel - Re: [PATCH 2/9]: Reduce Log I/O latency

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20071124184333.GK17536@waste.org>
Date:	Sat, 24 Nov 2007 12:43:33 -0600
From:	Matt Mackall <mpm@...enic.com>
To:	Andi Kleen <andi@...stfloor.org>
Cc:	David Chinner <dgc@....com>, Stewart Smith <stewart@...ql.com>,
	xfs-oss <xfs@....sgi.com>, lkml <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 2/9]: Reduce Log I/O latency

On Fri, Nov 23, 2007 at 01:01:15PM +0100, Andi Kleen wrote:
> On Fri, Nov 23, 2007 at 03:03:29PM +1100, David Chinner wrote:
> > On Fri, Nov 23, 2007 at 03:53:17AM +0100, Andi Kleen wrote:
> > > On Fri, Nov 23, 2007 at 12:15:39AM +1100, David Chinner wrote:
> > > > On Thu, Nov 22, 2007 at 01:06:11PM +0100, Andi Kleen wrote:
> > > > > > FWIW from a "real time" database POV this seems to make sense to me...
> > > > > > in fact, we probably rely on filesystem metadata way too much
> > > > > > (historically it's just "worked".... although we do seem to get issues
> > > > > > on ext3).
> > > > > 
> > > > > For that case you really would need priority inheritance: any metadata
> > > > > IO on behalf or blocking a process needs to use the process' block IO 
> > > > > priority.
> > > > 
> > > > How do you do that when the processes are blocking on semaphores,
> > > > mutexes or rw-semaphores in the fileysystem three layers removed from
> > > > the I/O in progress?
> > > 
> > > [...] I didn't say it was easy (or rather explicitely said it would be tricky).
> > > Probably it would be possible to fold it somehow into rt mutexes PI,
> > > but it's not easy and semaphores would need to be handled too.
> > > 
> > > Just my point was to solve the metadata RT problem unconditionally increasing
> > > the priority is a bad idea and not really a replacement to a "full"
> > > solution. Short term a user can just increase the priority of all the XFS 
> > > threads anyways.
> > 
> > The point is that it's not actually a thread-based problem - the priority
> > can't be inherited via the traditional mutex-like manner. There is no
> > connection between a thread and an I/o it has already issued and so you
> > can't transfer a priority from a blocked thread to an issued-but-blocked
> > i/o....
> 
> It could be handled in theory similar to standard CPU priority inheritance -- \
> keep track of IO priority of all threads you block and boost your IO priority
> always to that level. But it would be probably not very easy to do.

Well I think what Dave is saying is that we can't find the related
process. The submitter process may have even exited before the flush
happens.. You'd instead have to keep track of (the max of) all the
submitted I/O segment priorities related to the transaction instead.
But I'm sure there are complications.

-- 
Mathematics is the supreme nostalgia of our time.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/