linux-kernel - Re: [PATCH v3] scheduler: enhancement to show_state

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170809164250.aza4csoc2jcwkvx7@hirez.programming.kicks-ass.net>
Date:   Wed, 9 Aug 2017 18:42:50 +0200
From:   Peter Zijlstra <peterz@...radead.org>
To:     Yafang Shao <laoar.shao@...il.com>
Cc:     mingo@...hat.com, gregkh@...uxfoundation.org, jslaby@...e.com,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v3] scheduler: enhancement to show_state_filter and SysRq

On Wed, Aug 09, 2017 at 05:26:14PM +0800, Yafang Shao wrote:
> 2017-08-09 17:09 GMT+08:00 Peter Zijlstra <peterz@...radead.org>:
> > On Wed, Aug 09, 2017 at 04:01:49PM +0800, Yafang Shao wrote:
> >> 2017-08-09 15:43 GMT+08:00 Peter Zijlstra <peterz@...radead.org>:
> >> > On Wed, Aug 09, 2017 at 06:31:28PM +0800, Yafang Shao wrote:
> >> >> Sometimes we want to get tasks in TASK_RUNNING sepcifically,
> >> >> instead of dump all tasks.
> >> >>
> >> >> For example, when the loadavg are high, we want to dump
> >> >> tasks in TASK_RUNNING and TASK_UNINTERRUPTIBLE, which contribute
> >> >> to system load. But mostly there're lots of tasks in Sleep state,
> >> >> which occupies almost all of the kernel log buffer, even overflows
> >> >> it, that causes the useful messages get lost. Although we can
> >> >> enlarge the kernel log buffer, but that's not a good idea.
> >> >
> >> > That's what you have serial consoles for...
> >> >
> >> mostly we don't even have one console because we alwayas login the
> >> servers via ssh. And manage the servers with console is not so convenient.
> >
> > I find IPMI SOL very useful. Serial console (esp. earlyprintk) keeps on
> > working long after most other things have died.
> >
> > In any case, you can easily dump the printk output into your ssh session
> > if you want, use something like:
> >
> >         cat /dev/kmsg | tee logfile & echo t > /proc/sysrq-trigger
> 
> that's what I'm doing it currently :)
> Then I thought deeply why not do it more smartly?
> Introducing a new key(here I just modified the key 'w') only dump
> tasks in running and blocked should be more smarter.

Since you're strictly ssh based, you could maybe do a sysctl that allows
changing the 'default' filter of sysctl-t, dunno if that makes sense
though.

Also, since you're not actually debugging a dead machine, maybe you can
do a custom kernel module / systemtap / ebpf thing that collects
precisely the information you want.

sysrq is typically a last ditch debug mostly dead machine thing, which
you're very much not having.

> >
> > I really see no problem here. Then you can run a bit of awk or whatever
> > your favourite tool is to filter out the stuff you don't want.
> >
> Another question, if we could filter with scritpts in userland, why
> did we introduced the key 'w' to dump only blocked state
> as we already have a key 't' to dump all tasks ?

No idea that is long before my time, I expect because 'w' (blocked) is
typically a small number of tasks.