linux-kernel - Re: memcg creates an unkillable task in 3.2-rc2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20130729172046.GI22605@mtj.dyndns.org>
Date:	Mon, 29 Jul 2013 13:20:46 -0400
From:	Tejun Heo <tj@...nel.org>
To:	"Eric W. Biederman" <ebiederm@...ssion.com>
Cc:	Michal Hocko <mhocko@...e.cz>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	cgroups@...r.kernel.org, containers@...ts.linux-foundation.org,
	linux-kernel@...r.kernel.org, kent.overstreet@...il.com,
	Li Zefan <lizefan@...wei.com>,
	Glauber Costa <glommer@...il.com>,
	Johannes Weiner <hannes@...xchg.org>
Subject: Re: memcg creates an unkillable task in 3.2-rc2

Hey, Eric.

On Mon, Jul 29, 2013 at 10:03:35AM -0700, Eric W. Biederman wrote:
> So this is not a simple matter of a frozen task not dying when SIGKILL
> is received.  For the most part not dying when SIGKILL is received seems
> like correct behavior for a frozne task.  Certainly it is correct
> behavior for any other signal.
> 
> The issue is that the tasks don't freeze or that when thawed the SIGKILL
> is still ignored.  It seems a wake up is being missed in there somewhere.

That's actually interesting and shouldn't be happening.  Can you
please provide more data as to what's going on while freezing?  It's
likely that the problem is not caused by freezer per-se, the task
might be stuck elsewhere and just fails to reach the freezing point.

Would it be possible for memcg and freezer to deadlock?  Note that
while freezing is in progress, some tasks will enter freezer earlier
than others (of course) and won't respond to anything.  If memcg adds
wait dependency among the tasks being frozen, it'll surely deadlock.

> A single unified hierarchy is a really nasty idea for the same set of
> reasons. You have to recompile to disable a controller to see if it that
> controller's bugs are what are causing problems on your production
> system.  Compiles or even just a reboot is a very heavy hammer to ask
> people to use when they are triaging a problem.

For Nth time, unified hierarchy doesn't mean all controllers are
enabled on all hierarchies or that controllers can't be bound and
unbound dynamically.  Except for the removal of orthogonal
hierarchies, things actually become a lot more dynamic.

> I am also seeing what looks like a leak somewhere in the cgroup code as
> well.  After some runs of the same reproducer I get into a state where
> after everything is clean up.  All of the control groups have been
> removed and the cgroup filesystem is unmounted, I can mount a cgroup
> filesystem with that same combindation of subsystems, but I can't mount
> a cgroup filesystem with any of those subsystems in any other
> combination.  So I am guessing that the superblock is from the original
> mounting is still lingering for some reason.

Hmmm... yeah, if there are cgroups with refs remaining, that'd happen.
Note that AFAIU memcg keeps the cgroups hangling around until all the
pages are gone from it, so it could just be that it's still draining
which may take a long time.  Maybe dropping cache would work?

Thanks.

-- 
tejun
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/