[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date: Thu, 7 Jul 2016 14:39:37 +0200
From: Lars Ellenberg <lars.ellenberg@...bit.com>
To: NeilBrown <neilb@...e.com>
Cc: linux-block@...r.kernel.org, Jens Axboe <axboe@...nel.dk>,
linux-raid@...r.kernel.org, linux-kernel@...r.kernel.org,
"Martin K. Petersen" <martin.petersen@...cle.com>,
Mike Snitzer <snitzer@...hat.com>,
Peter Zijlstra <peterz@...radead.org>,
Jiri Kosina <jkosina@...e.cz>,
Ming Lei <ming.lei@...onical.com>,
linux-bcache@...r.kernel.org, Zheng Liu <gnehzuil.liu@...il.com>,
Keith Busch <keith.busch@...el.com>,
Takashi Iwai <tiwai@...e.de>, dm-devel@...hat.com,
Ingo Molnar <mingo@...hat.com>,
"Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
Shaohua Li <shli@...nel.org>,
Kent Overstreet <kent.overstreet@...il.com>,
Alasdair Kergon <agk@...hat.com>,
Roland Kammerer <roland.kammerer@...bit.com>
Subject: Re: [dm-devel] [RFC] block: fix blk_queue_split() resource exhaustion
On Thu, Jul 07, 2016 at 10:16:16AM +0200, Lars Ellenberg wrote:
> > > Instead, I suggest to distinguish between recursive calls to
> > > generic_make_request(), and pushing back the remainder part in
> > > blk_queue_split(), by pointing current->bio_lists to a
> > > struct recursion_to_iteration_bio_lists {
> > > struct bio_list recursion;
> > > struct bio_list remainder;
> > > }
> > >
> > > To have all bios targeted to drivers lower in the stack processed before
> > > processing the next piece of a bio targeted at the higher levels,
> > > as long as queued bios resulting from recursion are available,
> > > they will continue to be processed in FIFO order.
> > > Pushed back bio-parts resulting from blk_queue_split() will be processed
> > > in LIFO order, one-by-one, whenever the recursion list becomes empty.
> >
> > I really like this change. It seems to precisely address the problem.
> > The "problem" being that requests for "this" device are potentially
> > mixed up with requests from underlying devices.
> > However I'm not sure it is quite general enough.
> >
> > The "remainder" list is a stack of requests aimed at "this" level or
> > higher, and I think it will always exactly fit that description.
> > The "recursion" list needs to be a queue of requests aimed at the next
> > level down, and that doesn't quiet work, because once you start acting
> > on the first entry in that list, all the rest become "this" level.
>
> Uhm, well,
> that's how it has been since you introduced this back in 2007, d89d879.
> And it worked.
>
> > I think you can address this by always calling ->make_request_fn with an
> > empty "recursion", then after the call completes, splice the "recursion"
> > list that resulted (if any) on top of the "remainder" stack.
> >
> > This way, the "remainder" stack is always "requests for lower-level
> > devices before request for upper level devices" and the "recursion"
> > queue is always "requests for devices below the current level".
>
> Yes, I guess that would work as well,
> but may need "empirical proof" to check for performance regressions.
>
> > I also really *don't* like the idea of punting to a separate thread - it
> > seems to be just delaying the problem.
> >
> > Can you try move the bio_list_init(->recursion) call to just before
> > the ->make_request_fn() call, and adding
> > bio_list_merge_head(->remainder, ->recursion)
> > just after?
> > (or something like that) and confirm it makes sense, and works?
>
> Sure, will do.
Attached,
on top of the patch of my initial post.
Also fixes the issue for me.
> I'd suggest this would be a patch on its own though, on top of this one.
> Because it would change the order in which stacked bios are processed
> wrt the way it used to be since 2007 (my suggestion as is does not).
>
> Which may change performance metrics.
> It may even improve some of them,
> or maybe it does nothing, but we don't know.
>
> Thanks,
>
> Lars
>
View attachment "0001-block-generic_make_request-recursive-bios-process-de.patch" of type "text/x-diff" (5461 bytes)
Powered by blists - more mailing lists