linux-kernel - Re: [PATCH] x86/paravirt: Do not trace _paravirt_ident

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20160902104630.4b004e09@gandalf.local.home>
Date:   Fri, 2 Sep 2016 10:46:30 -0400
From:   Steven Rostedt <rostedt@...dmis.org>
To:     LKML <linux-kernel@...r.kernel.org>
Cc:     Jeremy Fitzhardinge <jeremy@...p.org>,
        Chris Wright <chrisw@...s-sol.org>,
        Alok Kataria <akataria@...are.com>,
        Rusty Russell <rusty@...tcorp.com.au>,
        virtualization@...ts.linux-foundation.org,
        Ingo Molnar <mingo@...nel.org>,
        "H. Peter Anvin" <hpa@...or.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Łukasz Daniluk <lukasz.daniluk@...el.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [PATCH] x86/paravirt: Do not trace _paravirt_ident_*()
 functions


I just spent half a day bisecting function tracing because I tripped
over this again. I thought this was merged, but I guess it was missed
again.

Can someone please pull this in. And mark it for stable, it goes
probably as far back as 2.6.32.

Thanks!

-- Steve

On Wed, 25 May 2016 13:47:26 -0400
Steven Rostedt <rostedt@...dmis.org> wrote:

> Łukasz Daniluk reported that on a RHEL kernel that his machine would lock up
> after enabling function tracer. I asked him to bisect the functions within
> available_filter_functions, which he did and it came down to three:
> 
>  _paravirt_nop(), _paravirt_ident_32() and _paravirt_ident_64()
> 
> It was found that this is only an issue when noreplace-paravirt is added to
> the kernel command line.
> 
> This means that those functions are most likely called within critical
> sections of the funtion tracer, and must not be traced.
> 
> In newer kenels _paravirt_nop() is defined within gcc asm(), and is no
> longer an issue. But both _paravirt_ident_{32,64}() causes the following
> splat when they are traced:
> 
>  mm/pgtable-generic.c:33: bad pmd ffff8800d2435150(0000000001d00054)
>  mm/pgtable-generic.c:33: bad pmd ffff8800d3624190(0000000001d00070)
>  mm/pgtable-generic.c:33: bad pmd ffff8800d36a5110(0000000001d00054)
>  mm/pgtable-generic.c:33: bad pmd ffff880118eb1450(0000000001d00054)
>  NMI watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [systemd-journal:469]
>  Modules linked in: e1000e
>  CPU: 2 PID: 469 Comm: systemd-journal Not tainted 4.6.0-rc4-test+ #513
>  Hardware name: Hewlett-Packard HP Compaq Pro 6300 SFF/339A, BIOS K01 v02.05 05/07/2012
>  task: ffff880118f740c0 ti: ffff8800d4aec000 task.ti: ffff8800d4aec000
>  RIP: 0010:[<ffffffff81134148>]  [<ffffffff81134148>] queued_spin_lock_slowpath+0x118/0x1a0
>  RSP: 0018:ffff8800d4aefb90  EFLAGS: 00000246
>  RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffff88011eb16d40
>  RDX: ffffffff82485760 RSI: 000000001f288820 RDI: ffffea0000008030
>  RBP: ffff8800d4aefb90 R08: 00000000000c0000 R09: 0000000000000000
>  R10: ffffffff821c8e0e R11: 0000000000000000 R12: ffff880000200fb8
>  R13: 00007f7a4e3f7000 R14: ffffea000303f600 R15: ffff8800d4b562e0
>  FS:  00007f7a4e3d7840(0000) GS:ffff88011eb00000(0000) knlGS:0000000000000000
>  CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
>  CR2: 00007f7a4e3f7000 CR3: 00000000d3e71000 CR4: 00000000001406e0
>  Stack:
>   ffff8800d4aefba0 ffffffff81cc5f47 ffff8800d4aefc60 ffffffff8122c15b
>   ffff8800d4aefcb0 ffff8800d4aefbd0 ffffffff811bf4cb 0000000000000002
>   0000000000000015 ffff8800d2276050 80000000c0fd8867 ffffea0000008030
>  Call Trace:
>   [<ffffffff81cc5f47>] _raw_spin_lock+0x27/0x30
>   [<ffffffff8122c15b>] handle_pte_fault+0x13db/0x16b0
>   [<ffffffff811bf4cb>] ? function_trace_call+0x15b/0x180
>   [<ffffffff8122ad85>] ? handle_pte_fault+0x5/0x16b0
>   [<ffffffff8122e322>] handle_mm_fault+0x312/0x670
>   [<ffffffff81231068>] ? find_vma+0x68/0x70
>   [<ffffffff810ab741>] __do_page_fault+0x1b1/0x4e0
>   [<ffffffff810aba92>] do_page_fault+0x22/0x30
>   [<ffffffff81cc7f68>] page_fault+0x28/0x30
>   [<ffffffff81574af5>] ? copy_user_enhanced_fast_string+0x5/0x10
>   [<ffffffff8129dec5>] ? seq_read+0x305/0x370
>   [<ffffffff81279668>] __vfs_read+0x28/0xe0
>   [<ffffffff81279645>] ? __vfs_read+0x5/0xe0
>   [<ffffffff81279645>] ? __vfs_read+0x5/0xe0
>   [<ffffffff81279df6>] vfs_read+0x86/0x130
>   [<ffffffff8127b216>] SyS_read+0x46/0xa0
>   [<ffffffff81cc6176>] entry_SYSCALL_64_fastpath+0x1e/0xa8
>  Code: 12 48 c1 ea 0c 83 e8 01 83 e2 30 48 98 48 81 c2 40 6d 01 00 48 03 14
>  c5 80 6a 5d 82 48 89 0a 8b 41 08 85 c0 75 09 f3 90 8b 41 08 <85> c0 74 f7
>  4c 8b 09 4d 85 c9 74 08 41 0f 18 09 eb 02 f3 90 8b
> 
> Reported-by: Łukasz Daniluk <lukasz.daniluk@...el.com>
> Signed-off-by: Steven Rostedt <rostedt@...dmis.org>
> 
> diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c
> index f08ac28b8136..f975d226be6e 100644
> --- a/arch/x86/kernel/paravirt.c
> +++ b/arch/x86/kernel/paravirt.c
> @@ -55,12 +55,12 @@ asm (".pushsection .entry.text, \"ax\"\n"
>       ".popsection");
>  
>  /* identity function, which can be inlined */
> -u32 _paravirt_ident_32(u32 x)
> +u32 notrace _paravirt_ident_32(u32 x)
>  {
>  	return x;
>  }
>  
> -u64 _paravirt_ident_64(u64 x)
> +u64 notrace _paravirt_ident_64(u64 x)
>  {
>  	return x;
>  }