linux-kernel - Re: [patch 0/7] improve memcg oom killer robustness v2

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20130918160304.6EDF2729@pobox.sk>
Date:	Wed, 18 Sep 2013 16:03:04 +0200
From:	"azurIt" <azurit@...ox.sk>
To:	Michal Hocko <mhocko@...e.cz>
Cc:	Johannes Weiner <hannes@...xchg.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	David Rientjes <rientjes@...gle.com>,
	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>,
	<linux-mm@...ck.org>, <cgroups@...r.kernel.org>, <x86@...nel.org>,
	<linux-arch@...r.kernel.org>, <linux-kernel@...r.kernel.org>
Subject: Re: [patch 0/7] improve memcg oom killer robustness v2

> CC: "Johannes Weiner" <hannes@...xchg.org>, "Andrew Morton" <akpm@...ux-foundation.org>, "David Rientjes" <rientjes@...gle.com>, "KAMEZAWA Hiroyuki" <kamezawa.hiroyu@...fujitsu.com>, "KOSAKI Motohiro" <kosaki.motohiro@...fujitsu.com>, linux-mm@...ck.org, cgroups@...r.kernel.org, x86@...nel.org, linux-arch@...r.kernel.org, linux-kernel@...r.kernel.org
>On Tue 17-09-13 13:15:35, azurIt wrote:
>[...]
>> Is something unusual on this stack?
>> 
>> 
>> [<ffffffff810d1a5e>] dump_header+0x7e/0x1e0
>> [<ffffffff810d195f>] ? find_lock_task_mm+0x2f/0x70
>> [<ffffffff810d1f25>] oom_kill_process+0x85/0x2a0
>> [<ffffffff810d24a8>] mem_cgroup_out_of_memory+0xa8/0xf0
>> [<ffffffff8110fb76>] mem_cgroup_oom_synchronize+0x2e6/0x310
>> [<ffffffff8110efc0>] ? mem_cgroup_uncharge_page+0x40/0x40
>> [<ffffffff810d2703>] pagefault_out_of_memory+0x13/0x130
>> [<ffffffff81026f6e>] mm_fault_error+0x9e/0x150
>> [<ffffffff81027424>] do_page_fault+0x404/0x490
>> [<ffffffff810f952c>] ? do_mmap_pgoff+0x3dc/0x430
>> [<ffffffff815cb87f>] page_fault+0x1f/0x30
>
>This is a regular memcg OOM killer. Which dumps messages about what is
>going to do. So no, nothing unusual, except if it was like that for ever
>which would mean that oom_kill_process is in the endless loop. But a
>single stack doesn't tell us much.
>
>Just a note. When you see something hogging a cpu and you are not sure
>whether it might be in an endless loop inside the kernel it makes sense
>to take several snaphosts of the stack trace and see if it changes. If
>not and the process is not sleeping (there is no schedule on the trace)
>then it might be looping somewhere waiting for Godot. If it is sleeping
>then it is slightly harder because you would have to identify what it is
>waiting for which requires to know a deeper context.
>-- 
>Michal Hocko
>SUSE Labs



I was finally able to get stack of problematic process :) I saved it two times from the same process, as Michal suggested (i wasn't able to take more). Here it is:

First (doesn't look very helpfull):
[<ffffffffffffffff>] 0xffffffffffffffff


Second:
[<ffffffff810e17d1>] shrink_zone+0x481/0x650
[<ffffffff810e2ade>] do_try_to_free_pages+0xde/0x550
[<ffffffff810e310b>] try_to_free_pages+0x9b/0x120
[<ffffffff81148ccd>] free_more_memory+0x5d/0x60
[<ffffffff8114931d>] __getblk+0x14d/0x2c0
[<ffffffff8114c973>] __bread+0x13/0xc0
[<ffffffff811968a8>] ext3_get_branch+0x98/0x140
[<ffffffff81197497>] ext3_get_blocks_handle+0xd7/0xdc0
[<ffffffff81198244>] ext3_get_block+0xc4/0x120
[<ffffffff81155b8a>] do_mpage_readpage+0x38a/0x690
[<ffffffff81155ffb>] mpage_readpages+0xfb/0x160
[<ffffffff811972bd>] ext3_readpages+0x1d/0x20
[<ffffffff810d9345>] __do_page_cache_readahead+0x1c5/0x270
[<ffffffff810d9411>] ra_submit+0x21/0x30
[<ffffffff810cfb90>] filemap_fault+0x380/0x4f0
[<ffffffff810ef908>] __do_fault+0x78/0x5a0
[<ffffffff810f2b24>] handle_pte_fault+0x84/0x940
[<ffffffff810f354a>] handle_mm_fault+0x16a/0x320
[<ffffffff8102715b>] do_page_fault+0x13b/0x490
[<ffffffff815cb87f>] page_fault+0x1f/0x30
[<ffffffffffffffff>] 0xffffffffffffffff


What do you think about it?

azur
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/