linux-kernel - Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87invr4tjm.fsf@notabene.neil.brown.name>
Date:	Wed, 27 Jul 2016 14:02:53 +1000
From:	NeilBrown <neilb@...e.com>
To:	Mikulas Patocka <mpatocka@...hat.com>
Cc:	Tetsuo Handa <penguin-kernel@...ove.SAKURA.ne.jp>,
	LKML <linux-kernel@...r.kernel.org>,
	Michal Hocko <mhocko@...nel.org>, linux-mm@...ck.org,
	dm-devel@...hat.com, Mel Gorman <mgorman@...e.de>,
	David Rientjes <rientjes@...gle.com>,
	Ondrej Kozina <okozina@...hat.com>,
	Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [dm-devel] [RFC PATCH 2/2] mm, mempool: do not throttle PF_LESS_THROTTLE tasks

On Tue, Jul 26 2016, Mikulas Patocka wrote:

> On Sat, 23 Jul 2016, NeilBrown wrote:
>
>> "dirtying ... from the reclaim context" ??? What does that mean?
>> According to
>>   Commit: 26eecbf3543b ("[PATCH] vm: pageout throttling")
>> From the history tree, the purpose of throttle_vm_writeout() is to
>> limit the amount of memory that is concurrently under I/O.
>> That seems strange to me because I thought it was the responsibility of
>> each backing device to impose a limit - a maximum queue size of some
>> sort.
>
> Device mapper doesn't impose any limit for in-flight bios.

I would suggest that it probably should. At least it should
"set_wb_congested()" when the number of in-flight bios reaches some
arbitrary threshold.

The write-back throttling needs this to get an estimate of how fast the
backing device is, so it can share the dirty_threshold space fairly
among the different backing devices.

I added an arbitrary limit to raid1 back in 2011 (34db0cd60f8a1f)
because the lack of a limit was causing problems.
Specifically the write queue would get so long that ext3 would block for
an extended period when trying to flush a transaction, and that blocked
lots of other things, like atime updates.

Maybe there have been other fixes since then to other parts of the
puzzle, but the congestion tracking still seems to be an important part
of the picture and I think it would be best if every bdi would admit to
being congested well before it has consumed a significant fraction of
memory in its output queue.

> I've made some patches that limit in-flight bios for device mapper in
> the past, but there were not integrated into upstream.

I second the motion to resurrect these.

Thanks,
NeilBrown

Download attachment "signature.asc" of type "application/pgp-signature" (819 bytes)