linux-kernel - Re: fix callgraphs of 32-bit processes on 64-bit kernels.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4B9E5EFF.1080308@gmail.com>
Date:	Mon, 15 Mar 2010 18:23:27 +0200
From:	Török Edwin <edwintorok@...il.com>
To:	Ingo Molnar <mingo@...hat.com>
CC:	Thomas Gleixner <tglx@...utronix.de>,
	"H. Peter Anvin" <hpa@...or.com>,
	Peter Zijlstra <a.p.zijlstra@...llo.nl>,
	Paul Mackerras <paulus@...ba.org>, x86@...nel.org,
	linux-kernel@...r.kernel.org
Subject: Re: fix callgraphs of 32-bit processes on 64-bit kernels.

On 03/15/2010 05:34 PM, Török Edwin wrote:
> Hi,
> 
> Callgraph profiling 32-bit apps on a 64-bit kernel doesn't work.
> The reason is that perf_callchain_user tries to read a stackframe with 64-bit
> pointers, which is wrong for a 32-bit process.
> 
> This patch fixes that, and I am almost able to get nice callgraph profiles
> from 32-bit apps now! (except for some problems with perf itself when tracing
> kernel modules, see [1])
> 
> Page-faults can be traced nicely (sid-ia32 is a 32-bit chroot):
> 
> $ sudo perf record -e page-faults -f -g /home/edwin/sid-ia32/usr/bin/glxgears
> $ sudo perf report
> ...
>     45.33%  libc-2.10.2.so                   [.] __GI_memcpy
>             |
>             --- __GI_memcpy
>                 _mesa_BufferDataARB
>                 _mesa_meta_Clear
>                 radeonUserClear
>                 r700Clear
>                 _mesa_Clear
>                 0x8049367
>                 0x804a6ba
>                 __libc_start_main
>                 0x8049111
> 
>     16.96%  libc-2.10.2.so                   [.] __GI_memset
>             |
>             --- __GI_memset
>                 _tnl_init_vertices
>                 _swsetup_CreateContext
>                 r600CreateContext
>                 driCreateNewContext
>                 dri2CreateNewContext
>                 0xf77ab7dd
>                 0xf7783c67
>                 0xf778514c
>                 0x804974f
>                 0x804a33d
>                 __libc_start_main
>                 0x8049111
> 
> And CPU cycles can be traced too in userspace:
> $ sudo perf record -f -g /home/edwin/sid-ia32/usr/bin/glxgears
> $ sudo perf report --sort comm,dso
>     [...]
>     44.97%  glxgears  r600_dri.so
>             |
>             |--5.85%-- r700SendSPIState
>             |          radeonEmitState
>             |          r700DrawPrims
>             |          |
>             |          |--95.45%-- vbo_save_playback_vertex_list
>             |          |          execute_list
>             |          |          _mesa_CallList
>             |          |          neutral_CallList
>             |          |          |
>             |          |          |--38.10%-- 0x80494a8
>             |          |          |          0x804a6ba
>             |          |          |          __libc_start_main
>             |          |          |          0x8049111
>     [....]
>     40.00%  glxgears  [kernel]
>             |
>             |--3.14%-- copy_user_generic_string
>             |          |
>             |          |--71.70%-- 0xffffffffa01b4493
>             |          |          0xffffffffa01b7c0b
>             |          |          0xffffffffa018b45b
>             |          |          0xffffffffa00ca927
>             |          |          0xffffffffa01c524e
>             |          |          compat_sys_ioctl
>             |          |          sysenter_dispatch
>             |          |          0xf77ca430
>             |          |          drmCommandWriteRead
>             |          |          0xf74d7ab5
>             |          |          0xf74d89a4
>             |          |          rcommonFlushCmdBufLocked
>             |          |          rcommonFlushCmdBuf
>             |          |          radeonFlush
>             |          |          _mesa_flush
>             |          |          _mesa_Flush
>             |          |          0xf775f270
>             |          |          0x804a6d5
>             |          |          __libc_start_main
>             |          |          0x8049111
>             |          |
>             |          |--15.09%-- 0xffffffffa01c524e
>             |          |          compat_sys_ioctl
>             |          |          sysenter_dispatch
>             |          |          0xf77ca430
>             |          |          drmCommandWriteRead
> 
> [1] But there is a problem with the perf tool: it can't trace addresses in
> kernel modules. This is a problem regardless if the traced app is 32-bit or
> 64-bit; and regardless if I do callgraph profiling or not.
> See the above trace, where the kernel addresses have all ffffffffa0* without a
> symbol name.
> 
> If I look at /proc/kallsyms I can guess the symbols, for example
> 0xffffffffa01b4493 is probably this one:
>   ffffffffa01b4411 t r600_cs_packet_parse	[radeon]
> 
> If I record/report without callgraph its the same problem:
> [...]
>     24.01%  glxgears  [kernel]                           [k] 0xffffffffa01b4ee9
>      3.96%  glxgears  libdrm_radeon.so.1.0.0             [.] cs_gem_write_reloc
>      3.53%  glxgears  r600_dri.so                        [.] r700SendSPIState
>      2.77%  glxgears  r600_dri.so                        [.] r700DrawPrims
>      1.99%  glxgears  r600_dri.so                        [.] r700SendVSConsts
> 
> Kernel symbol for 0xffffffffa01b4ee9 is not shown, I can guess it is this one
> (hey it was an exact match!):
>   ffffffffa01b4ee9 t r600_packet3_check	[radeon]
> 
> It would be good if perf knew how to lookup symbols in kernel modules!

BTW perf report -m -k /home/edwin/builds/linux-2.6/vmlinux doesn't show
the symbols either.

> 
> Best regards,
> --Edwin

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/