linux-kernel - [PATCHSET 2/3 block/for-4.1/core] writeback: cgroup writeback backpressure propagation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <1427087267-16592-1-git-send-email-tj@kernel.org>
Date:	Mon, 23 Mar 2015 01:07:29 -0400
From:	Tejun Heo <tj@...nel.org>
To:	axboe@...nel.dk
Cc:	linux-kernel@...r.kernel.org, jack@...e.cz, hch@...radead.org,
	hannes@...xchg.org, linux-fsdevel@...r.kernel.org,
	vgoyal@...hat.com, lizefan@...wei.com, cgroups@...r.kernel.org,
	linux-mm@...ck.org, mhocko@...e.cz, clm@...com,
	fengguang.wu@...el.com, david@...morbit.com, gthelen@...gle.com
Subject: [PATCHSET 2/3 block/for-4.1/core] writeback: cgroup writeback backpressure propagation

Hello,

While the previous patchset[2] implemented cgroup writeback support,
the IO back pressure propagation mechanism implemented in
balance_dirty_pages() and its subroutines isn't yet aware of cgroup
writeback.

Processes belonging to a memcg may have access to only subset of total
memory available in the system and not factoring this into dirty
throttling rendered it completely ineffective for processes under
memcg limits and memcg ended up building a separate ad-hoc degenerate
mechanism directly into vmscan code to limit page dirtying.

This patchset refactors the dirty throttling logic implemented in
balance_dirty_pages() and its subroutines os that it can handle both
global and memcg memory domains.  Dirty throttling mechanism is
applied against both the global and memcg constraints and the more
restricted of the two is used for actual throttling.

This makes the dirty throttling mechanism operational for memcg
domains including writeback-bandwidth-proportional dirty page
distribution inside them.

This patchset contains the following 18 patches.

 0001-memcg-make-mem_cgroup_read_-stat-event-iterate-possi.patch
 0002-writeback-reorganize-__-wb_update_bandwidth.patch
 0003-writeback-implement-wb_domain.patch
 0004-writeback-move-global_dirty_limit-into-wb_domain.patch
 0005-writeback-consolidate-dirty-throttle-parameters-into.patch
 0006-writeback-add-dirty_throttle_control-wb_bg_thresh.patch
 0007-writeback-make-__wb_dirty_limit-take-dirty_throttle_.patch
 0008-writeback-add-dirty_throttle_control-pos_ratio.patch
 0009-writeback-add-dirty_throttle_control-wb_completions.patch
 0010-writeback-add-dirty_throttle_control-dom.patch
 0011-writeback-make-__wb_writeout_inc-and-hard_dirty_limi.patch
 0012-writeback-separate-out-domain_dirty_limits.patch
 0013-writeback-move-over_bground_thresh-to-mm-page-writeb.patch
 0014-writeback-update-wb_over_bg_thresh-to-use-wb_domain-.patch
 0015-writeback-implement-memcg-wb_domain.patch
 0016-writeback-reset-wb_domain-dirty_limit-_tstmp-when-me.patch
 0017-writeback-implement-memcg-writeback-domain-based-thr.patch
 0018-mm-vmscan-remove-memcg-stalling-on-writeback-pages-d.patch

0001-0002 are prep patches.

0003-0014 refactors dirty throttling logic so that it operates on
wb_domain.

0015-0018 implement memcg wb_domain.

This patchset is on top of

  block/for-4.1/core bfd343aa1718 ("blk-mq: don't wait in blk_mq_queue_enter() if __GFP_WAIT isn't set")
+ [1] [PATCH] writeback: fix possible underflow in write bandwidth calculation
+ [2] [PATCHSET 1/3 v2 block/for-4.1/core] writeback: cgroup writeback support

and available in the following git branch.

 git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup.git review-cgroup-writeback-backpressure-20150322

diffstat follows.  Thanks.

 fs/fs-writeback.c                |   32 -
 include/linux/backing-dev-defs.h |    1 
 include/linux/memcontrol.h       |   21 +
 include/linux/writeback.h        |   82 +++-
 include/trace/events/writeback.h |    7 
 mm/backing-dev.c                 |    9 
 mm/memcontrol.c                  |  145 +++++--
 mm/page-writeback.c              |  722 +++++++++++++++++++++++++--------------
 mm/vmscan.c                      |  109 +----
 9 files changed, 716 insertions(+), 412 deletions(-)

--
tejun

[L] http://lkml.kernel.org/g/1420579582-8516-1-git-send-email-tj@kernel.org
[1] http://lkml.kernel.org/g/20150323041848.GA8991@htj.duckdns.org
[2] http://lkml.kernel.org/g/1427086499-15657-1-git-send-email-tj@kernel.org
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/