linux-kernel - Re: [PATCH v6 0/9] memcg: per cgroup dirty page accounting

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 17 Mar 2011 14:15:43 -0400
From:	Vivek Goyal <vgoyal@...hat.com>
To:	Jan Kara <jack@...e.cz>
Cc:	Greg Thelen <gthelen@...gle.com>,
	Johannes Weiner <hannes@...xchg.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	containers@...ts.osdl.org, linux-fsdevel@...r.kernel.org,
	Andrea Righi <arighi@...eler.com>,
	Balbir Singh <balbir@...ux.vnet.ibm.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	Daisuke Nishimura <nishimura@....nes.nec.co.jp>,
	Minchan Kim <minchan.kim@...il.com>,
	Ciju Rajan K <ciju@...ux.vnet.ibm.com>,
	David Rientjes <rientjes@...gle.com>,
	Wu Fengguang <fengguang.wu@...el.com>,
	Chad Talbott <ctalbott@...gle.com>,
	Justin TerAvest <teravest@...gle.com>,
	Curt Wohlgemuth <curtw@...gle.com>
Subject: Re: [PATCH v6 0/9] memcg: per cgroup dirty page accounting

On Thu, Mar 17, 2011 at 06:59:08PM +0100, Jan Kara wrote:
> On Thu 17-03-11 13:12:19, Vivek Goyal wrote:
> > On Thu, Mar 17, 2011 at 03:46:41PM +0100, Jan Kara wrote:
> > [..]
> > > > - bdi writeback: will revert some of the mmotm memcg dirty limit changes to
> > > >   fs-writeback.c so that wb_do_writeback() will return to checking
> > > >   wb_check_background_flush() to check background limits and being
> > > > interruptible if
> > > >   sync flush occurs.  wb_check_background_flush() will check the global
> > > >   memcg_over_bg_limit list for memcg that are over their dirty limit.
> > > >   wb_writeback() will either (I am not sure):
> > > >   a) scan memcg's bdi_memcg list of inodes (only some of them are dirty)
> > > >   b) scan bdi dirty inode list (only some of them in memcg) using
> > > >      inode_in_memcg() to identify inodes to write.  inode_in_memcg(inode,memcg),
> > > >      would walk memcg- -> memcg_bdi -> memcg_mapping to determine if the memcg
> > > >      is caching pages from the inode.
> > > Hmm, both has its problems. With a) we could queue all the dirty inodes
> > > from the memcg for writeback but then we'd essentially write all dirty data
> > > for a memcg, not only enough data to get below bg limit. And if we started
> > > skipping inodes when memcg(s) inode belongs to get below bg limit, we'd
> > > risk copying inodes there and back without reason, cases where some inodes
> > > never get written because they always end up skipped etc. Also the question
> > > whether some of the memcgs inode belongs to is still over limit is the
> > > hardest part of solution b) so we wouldn't help ourselves much.
> > 
> > May be I am missing something but can't we just start traversing
> > through list of memcg_over_bg_list and take option a) to traverse
> > through list of inodes and write them till we are below limit of
> > that group. We of course skip inodes which are not dirty.
> > 
> > This is assuming that root group is also part of that list so that
> > inodes in root group do not starve writeback.
> > 
> > We still continue to have all the inodes on bdi wb structure and
> > memcg will just give us pointers to those inodes. So for background
> > write, instead of going serially through dirty inodes list, we
> > will first pick the cgroup to write and then inode to write. As
> > we will be doing round robin among cgroup list, it will make sure
> > that none of the cgroups (including root) as well as inode are not
> > starved.
>   I was considering this as well and didn't quite like it but on a second
> thought it need not be that bad. If we wrote MAX_WRITEBACK_PAGES from one
> memcg, then switched to another one while keeping pointers to per-memcg inode
> list (for the time when we return to this memcg), it could work just fine.

Yes, we can write MAX_WRITEBACK_PAGES from each memcg and then move on to
next one. In fact memcg_bdi should have list of memcg_mapping. So once we
select the inode (memcg_mapping) from cgroup for writeout (move inode
on ->b_io list), we can also shuffle the position of memcg_mapping with-in
memory cgroup so that inodes with-in a cgroup get fair share of writeout in
a round robin manner.

As you said in other mail, we probably will keep MEMCG_BDI_WRITTEN count
so that IO less throttling can distribute the pages completed to right
cgroup. 

Down the line we can probably also maintain MEMCG_BDI_WRITEBACK to keep track
how many pages are already under writeout from a cgroup and skip that cgroup
if too many pages are already in-flight. This might help us push more
WRITES for higher weight IO cgroup as compared to lower weight IO cgroup.

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/