linux-kernel - Re: [QUESTION] Is there a better way to get ftrace dump on guest?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160629005741.GA8844@bbox>
Date:	Wed, 29 Jun 2016 09:57:41 +0900
From:	Minchan Kim <minchan@...nel.org>
To:	Rabin Vincent <rabin@....in>
CC:	Namhyung Kim <namhyung@...nel.org>, <kvm@...r.kernel.org>,
	Steven Rostedt <rostedt@...dmis.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [QUESTION] Is there a better way to get ftrace dump on guest?

Hello,

On Tue, Jun 28, 2016 at 06:46:34PM +0200, Rabin Vincent wrote:
> On Tue, Jun 28, 2016 at 03:33:18PM +0900, Namhyung Kim wrote:
> > On Tue, Jun 28, 2016 at 3:25 PM, Namhyung Kim <namhyung@...nel.org> wrote:
> > > I'm running some guest machines for kernel development.  For debugging
> > > purpose, I use lots of trace_printk() since it's faster than normal
> > > printk().  When kernel crash happens the trace buffer is printed on
> > > console (I set ftrace_dump_on_oops) but it takes too much time.  I
> > > don't want to reduce the size of ring buffer as I want to collect the
> > > debug info as much as possible.  And I also want to see trace from all
> > > cpu so 'ftrace_dump_on_oop = 2' is not an option.
> > >
> > > I know the kexec/kdump (and the crash tool) can dump and analyze the
> > > trace buffer later.  But it's cumbersome to do it everytime and more
> > > importantly, I don't want to spend the memory for the crashkernel.
> 
> Assuming you're using QEMU:
> 
> QEMU has a dump-guest-memory command which can be used to dump the
> guest's entire memory to an ELF which can be loaded by the crash utility
> to extract the trace buffer.  This doesn't require kexec/kdump or any
> other support from the guest kernel.

Thanks for the hint. It's surely handy rather than kexec/kdump.

A question is that it's possible to capture guest's entire memory
when guest kernel is oops?
I mean I don't want to capture alive guest but get snapshot image
when guest kernel encounters BUG_ON and see event trace from the
image.

Anyway, I tried crashtool and load trace.so but failed to load
extension module 'trace.so' because read_string failed in
ftrace_get_event_type_name of trace.c.
Does it work with recent kernel?

My kernel is 4.7.0-rc4-mm1.

> 
> It's apparently even possible to run QEMU with the guest memory in a
> file and load that to crash directly, although this is not something
> I've had a chance to try out myself:
> 
> https://github.com/crash-utility/crash/commit/89ed9d0a7f7da4578294a492c1ad857244ce7352