linux-kernel - Re: [PATCH 10/10] mm: per device dirty threshold

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1177406817.26937.65.camel@twins>
Date:	Tue, 24 Apr 2007 11:26:57 +0200
From:	Peter Zijlstra <a.p.zijlstra@...llo.nl>
To:	Miklos Szeredi <miklos@...redi.hu>
Cc:	neilb@...e.de, linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	akpm@...ux-foundation.org, dgc@....com,
	tomoki.sekiyama.qu@...achi.com, nikita@...sterfs.com,
	trond.myklebust@....uio.no, yingchao.zhou@...il.com
Subject: Re: [PATCH 10/10] mm: per device dirty threshold

On Tue, 2007-04-24 at 11:14 +0200, Miklos Szeredi wrote:

> > > I'm still not quite sure what purpose the above "soft" limiting
> > > serves.  It seems to just give advantage to writers, which managed to
> > > accumulate lots of dirty pages, and then can convert that into even
> > > more dirtyings.
> > 
> > The queues only limit the actual in-flight writeback pages,
> > balance_dirty_pages() considers all pages that might become writeback as
> > well as those that are.
> > 
> > > Would it make sense to remove this behavior, and ensure that
> > > balance_dirty_pages() doesn't return until the per-queue limits have
> > > been complied with?
> > 
> > I don't think that will help, balance_dirty_pages drives the queues.
> > That is, it converts pages from mere dirty to writeback.
> 
> Yes.  But current logic says, that if you convert "write_chunk" dirty
> to writeback, you are allowed to dirty "ratelimit" more. 
> 
> D: number of dirty pages
> W: number of writeback pages
> L: global limit
> C: write_chunk = ratelimit_pages * 1.5
> R: ratelimit
> 
> If D+W >= L, then R = 8
> 
> Let's assume, that D == L and W == 0.  And that all of the dirty pages
> belong to a single device.  Also for simplicity, lets assume an
> infinite length queue, and a slow device.
> 
> Then while converting the dirty pages to writeback, D / C * R new
> dirty pages can be created.  So when all existing dirty have been
> converted:
> 
>   D = L / C * R
>   W = L
> 
>   D + W = L * (1 + R / C)
> 
> So we see, that we're now even more above the limit than before the
> conversion.  This means, that we starve writers to other devices,
> which don't have as many dirty pages, because until the slow device
> doesn't finish these writes they will not get to do anything.
> 
> Your patch helps this in that if the other writers have an empty queue
> and no dirty, they will be allowed to slowly start writing.  But they
> will not gain their full share until the slow dirty-hog goes below the
> global limit, which may take some time.
> 
> So I think the logical thing to do, is if the dirty-hog is over it's
> queue limit, don't let it dirty any more until it's dirty+writeback go
> below the limit.  That allowes other devices to more quickly gain
> their share of dirty pages.

Ahh, now I see; I had totally blocked out these few lines:

			pages_written += write_chunk - wbc.nr_to_write;
			if (pages_written >= write_chunk)
				break;		/* We've done our duty */

yeah, those look dubious indeed... And reading back Neil's comments, I
think he agrees.

Shall we just kill those?

-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/