Message-ID: <20080717134736.GA11420@redhat.com>
Date:	Thu, 17 Jul 2008 09:47:36 -0400
From:	Vivek Goyal <vgoyal@...hat.com>
To:	Kazunaga Ikeno <k-ikeno@...jp.nec.com>
Cc:	"'David Collier-Brown'" <davecb@....com>,
	"'KAMEZAWA Hiroyuki'" <kamezawa.hiroyu@...fujitsu.com>,
	"'Rik van Riel'" <riel@...hat.com>,
	"'Paul Menage'" <menage@...gle.com>,
	"'linux kernel mailing list'" <linux-kernel@...r.kernel.org>,
	"'Libcg Devel Mailing List'" <libcg-devel@...ts.sourceforge.net>,
	"'Balbir Singh'" <balbir@...ux.vnet.ibm.com>,
	"'Dhaval Giani'" <dhaval@...ux.vnet.ibm.com>,
	"'Peter Zijlstra'" <pzijlstr@...hat.com>,
	"'Morton Andrew Morton'" <akpm@...ux-foundation.org>,
	"'Thomas Graf'" <tgraf@...hat.com>,
	"'Ulrich Drepper'" <drepper@...hat.com>
Subject: Re: [RFC] How to handle the rules engine for cgroups

On Thu, Jul 17, 2008 at 04:05:17PM +0900, Kazunaga Ikeno wrote:
> Vivek Goyal wrote:
> > On Mon, Jul 14, 2008 at 10:44:43AM -0400, David Collier-Brown wrote:
> > > Vivek Goyal wrote:
> > >> If admin has decided to group applications and has written the rules for
> > >> it then applications should not know anything about grouping. So I think
> > >> application writing an script for being placed into the right group should
> > >> be out of question. Now how does an admin write a wrapper around existing
> > >> application without breaking anything else.
> > >
> > > In the Solaris world, processes are placed into cgroups (projects) by
> > > one of two mechanisms:
> > >
> > > 1) inheritance, with everything I create in my existing project.
> > >   To get this started, there is a mechanism under login/getty/whatever
> > > reading the /etc/projects file and, for example, tossing user davecb
> > > into a "user.davecb" project.
> > >
> > 
> > Placing the login sessions in right cgroup based on uid/gid rules is
> > probably easy as check needs to be placed only on system entry upon login
> > (Pam plugin should do).  And after that any job started by the user
> > will automatically start in the same cgroup.
> > 
> > > 2) explicit placement with newtask, which starts a program or moves
> > >   a process into a project/cgroup
> > >
> > 
> > explicit placement of task based on application type will be tricky.
> > 
> > > I have a "bg" project which I use for limiting resource consumption of
> > > background jobs, and a background command which either starts or moves
> > > jobs, thusly:
> > >
> > > case "$1" in
> > > [0-9]*) # It's a pid
> > >         newtask -p bg -c $1
> > 
> > Ok, this is moving of tasks from one cgroup to other based on pid. This
> > is really easy to do through cgroup file system. Just a matter of writing
> > to task file.
> > 
> > >         ;;
> > >  *) # It's a command-line
> > >         newtask -p bg "$@" &
> > >         ;;
> > 
> > So here a user explicitly invokes the wrapper passing it the targeted
> > cgroup and the application to be launched in that cgroup. This should work
> > if there is a facility if user has created its own cgroups (lets say
> > under user controlled cgroup dir in the hierarchy) and user explicitly
> > wants to control the resources of applications under its dir. For example,
> > 
> >  		/mnt/cgroup
> > 		|	|
> > 		gid1	gid2
> > 		|  |	|  |
> > 	      uid1 uid2	uid3 uid4
> > 	     |  |
> > 	 proj1  proj2
> > 
> > Here probably admin can write the rules for how users are allocated the
> > resources and give ability to users to create subdirs under their cgroups
> > where users can create more cgroups and can do their own resource
> > management based on application tasks and place applications in the right
> > cgroup by writing wrappers as mentioned by you "newtask".
> > 
> > But here there is no discrimination of application type by admin. Admin
> > controls resource divisions only based on uid/gid. And users can manage
> > applications within their user groups. In fact I am having hard time thinking
> > in what kind of scenarios, there is a need for an admin to control
> > resource based on application type? Do we really need setups like, on
> > a system databases should get network bandwidth of 30%. If yes, then
> > it becomes tricky where admin need to write a wrapper to place the task
> > in right cgroup without application/user knowing it.
> 
> I think a wrapper (move to right group and calls exec) will run by user, not by admin.
> In explicit placement, user knows what a type of application he/she launch.
> 
>  		/mnt/cgroup
> 		|	|
> 		gid1	gid2
> 		|  |	|  |
> 	      uid1 uid2	uid3 uid4
> 	     |  |
> 	 proj1  proj2
> 

This is the easy to handle situation and I am hoping it will work in many
of the cases.

Currently I am writting a patch for libcg which allows querying the
destination cgroup based on uid/gid and libcg will also migrate the
application there. I am also writing a pam plugin which will move
all the login sessions to respective cgroup (as mentioned by rule file).
Will also modify "init" so that all the system services to into cgroup
belonging to root.

Once user is logged in and running into his resource group, he can manage
further subgroups at his own based on his application requirements (as you
mentioned proj1 and proj2 here).

> [uid1/gid1]% newtask.sh proj1app
> ... proj1app run under /mnt/cgroup/gid1/uid1
> 

Yes, so if a user does not specifically launch an application targetted
for a particular cgroup, then it will run into default group for that
user (as specified by rule file). In this case under /mnt/cgroup/gid1/uid1.

> [uid1/gid1]% newtask.sh --type proj1type proj1app
> ... proj1app run under /mnt/cgroup/gid1/uid1/proj1
>  

IOW, probably a user can say.

newtask.sh --cgrp /mnt/cgroup/gid1/uid1/proj1/  proj1app

> In this case, admin sets up limitation of proj1type.

I think admin should setup the limits only til /mnt/cgroup/gid1/uid1.
After that how resources allocated to uid1 are subdivided between various
user applications should be controller by user. So resources under
proj1 and proj2 will be full controlled by user.

> Also I guess proj1type has ownership (ex: proj1type allows gid1).
> Isn't this enough?

I think to begin with and to get some kind of simple functionality
going it might be good. I am sure others will target for more complex
configurations and usages.

Thanks
Vivek
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists