Message-ID: <CAEf4BzY9diEi2_tHsLxB4Yk-ZAWHT=XJNmagjQtOXc7qShqgrA@mail.gmail.com>
Date: Thu, 1 Aug 2024 14:13:41 -0700
From: Andrii Nakryiko <andrii.nakryiko@...il.com>
To: Peter Zijlstra <peterz@...radead.org>, Adrian Hunter <adrian.hunter@...el.com>
Cc: Oleg Nesterov <oleg@...hat.com>, andrii@...nel.org, mhiramat@...nel.org, 
	jolsa@...nel.org, rostedt@...dmis.org, linux-kernel@...r.kernel.org, 
	linux-trace-kernel@...r.kernel.org, bpf <bpf@...r.kernel.org>
Subject: Re: [PATCH v4 0/9] uprobes: misc cleanups/simplifications

On Thu, Aug 1, 2024 at 11:58 AM Andrii Nakryiko
<andrii.nakryiko@...il.com> wrote:
>
> + bpf
>
> On Thu, Aug 1, 2024 at 6:36 AM Peter Zijlstra <peterz@...radead.org> wrote:
> >
> > On Thu, Aug 01, 2024 at 03:26:38PM +0200, Oleg Nesterov wrote:
> > > (Andrii, I'll try to look at your new series on Weekend).
> >
> > OK, I dropped all your previous patches and stuffed these in.
> >
> > They should all be visible in queue/perf/core, and provided the robot
> > doesn't scream, I'll push them into tip/perf/core soonish.
>
> Just FYI, it seems like tip/perf/core is currently broken for uprobes
> (and by implication also queue/perf/core). Also torvalds/linux/master
> master is broken. See what I'm getting when running BPF selftests
> dealing with uprobes. Sometimes I only get that WARNING and nothing
> else.
>
> I'm bisecting at the moment with bpf/master being a "good" checkpoint,
> will let you know once I bisect.

Ok, this bisected to:

675ad74989c2 ("perf/core: Add aux_pause, aux_resume, aux_start_paused")

Reverting all (applied to tip/perf/core) four patches from that series:

6763ebdb4983 (tip/perf/core) perf/x86/intel: Do not enable large PEBS
for events with aux actions or aux sampling
6a45d8847597 perf/x86/intel/pt: Add support for pause / resume
675ad74989c2 perf/core: Add aux_pause, aux_resume, aux_start_paused
d92792a4b26e perf/x86/intel/pt: Fix sampling synchronization

... makes everything work again. I'll leave it up to you and Adrian to
figure this out.

>
> [   34.343557] ------------[ cut here ]------------
> [   34.343906] WARNING: CPU: 3 PID: 2364 at
> kernel/trace/trace_uprobe.c:1109 __probe_event_disable+0x26/0x80
> [   34.344468] Modules linked in:
> [   34.344488] BUG: unable to handle page fault for address: ffffc90001c5bea0
> [   34.345071] #PF: supervisor read access in kernel mode
> [   34.345370] #PF: error_code(0x0000) - not-present page
> [   34.345700] PGD 100000067 P4D 100000067 PUD ffff88810d86cd40
> [   34.346061] Oops: Oops: 0000 [#1] PREEMPT SMP NOPTI
> [   34.346377] CPU: 3 UID: 0 PID: 2364 Comm: test_progs Tainted: G
>       OE      6.11.0-rc1-00006-g6763ebdb4983 #115
> [   34.347052] Tainted: [O]=OOT_MODULE, [E]=UNSIGNED_MODULE
> [   34.347392] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS rel-1.14.0-0-g155821a1990b-prebuilt.qemu.org 04
> [   34.348085] RIP: 0010:__wake_up_common+0x3e/0xc0
> [   34.348359] Code: 89 d4 55 53 48 89 fb 48 83 ec 08 8b 05 c3 05 a8
> 02 89 74 24 04 85 c0 75 6d 48 8b 43 48 48 83 c3 48 4f
> [   34.349440] RSP: 0018:ffffc900001d0d90 EFLAGS: 00010087
> [   34.349796] RAX: ffffc90001c5bea0 RBX: ffff88810138e0e8 RCX: 0000000000000001
> [   34.350282] RDX: 0000000080010005 RSI: ffffffff8295f82c RDI: ffffc90001c5be88
> [   34.350768] RBP: ffff88810138e0a0 R08: 0000000000000000 R09: 000000000000438f
> [   34.351245] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000001
> [   34.351703] R13: 0000000000000046 R14: 0000000000000000 R15: 0000000000000000
> [   34.352112] FS:  00007fd71c7b3d00(0000) GS:ffff88881ca00000(0000)
> knlGS:0000000000000000
> [   34.352574] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   34.352911] CR2: ffffc90001c5bea0 CR3: 000000010b456005 CR4: 0000000000370ef0
> [   34.353320] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   34.353734] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   34.354144] Call Trace:
> [   34.354290]  <IRQ>
> [   34.354413]  ? __die+0x1f/0x60
> [   34.354601]  ? page_fault_oops+0x14c/0x450
> [   34.354848]  ? search_bpf_extables+0xa8/0x150
> [   34.355105]  ? fixup_exception+0x22/0x2d0
> [   34.355342]  ? exc_page_fault+0x207/0x210
> [   34.355579]  ? asm_exc_page_fault+0x22/0x30
> [   34.355829]  ? __wake_up_common+0x3e/0xc0
> [   34.356065]  __wake_up+0x32/0x60
> [   34.356261]  ep_poll_callback+0x13e/0x270
> [   34.356502]  __wake_up_common+0x7d/0xc0
> [   34.356731]  __wake_up+0x32/0x60
> [   34.356961]  irq_work_single+0x67/0x90
> [   34.357184]  irq_work_run_list+0x26/0x40
> [   34.357442]  update_process_times+0x83/0xa0
> [   34.357717]  tick_nohz_handler+0x97/0x140
> [   34.357977]  ? __pfx_tick_nohz_handler+0x10/0x10
> [   34.358283]  __hrtimer_run_queues+0x19a/0x3b0
> [   34.358580]  hrtimer_interrupt+0xfe/0x240
> [   34.358864]  __sysvec_apic_timer_interrupt+0x87/0x210
> [   34.359189]  sysvec_apic_timer_interrupt+0x98/0xc0
> [   34.359487]  </IRQ>
> [   34.359614]  <TASK>
> [   34.359745]  asm_sysvec_apic_timer_interrupt+0x16/0x20
> [   34.360042] RIP: 0010:print_modules+0x27/0xd0
> [   34.360301] Code: 90 90 90 0f 1f 44 00 00 53 48 c7 c7 63 22 98 82
> 48 83 ec 18 e8 da 8d fc ff bf 01 00 00 00 e8 80 d1 f1
> [   34.361372] RSP: 0018:ffffc90001e5bc58 EFLAGS: 00000297
> [   34.361682] RAX: ffffffffa03f0948 RBX: 0000000000000000 RCX: 0000000000000000
> [   34.362091] RDX: 0000000000000002 RSI: 0000000000000027 RDI: 0000000000000001
> [   34.362502] RBP: ffffc90001e5bd28 R08: 00000000fffeffff R09: 0000000000000001
> [   34.362917] R10: 0000000000000000 R11: ffffffff83299920 R12: ffffffff81233536
> [   34.363325] R13: 0000000000000009 R14: 0000000000000455 R15: ffffffff8298f924
> [   34.363740]  ? __probe_event_disable+0x26/0x80
> [   34.364009]  ? print_modules+0x20/0xd0
> [   34.364230]  ? __probe_event_disable+0x26/0x80
> [   34.364488]  __warn+0x6f/0x180
> [   34.364683]  ? __probe_event_disable+0x26/0x80
> [   34.364945]  report_bug+0x18d/0x1c0
> [   34.365156]  handle_bug+0x3a/0x70
> [   34.365354]  exc_invalid_op+0x13/0x60
> [   34.365571]  asm_exc_invalid_op+0x16/0x20
> [   34.365838] RIP: 0010:__probe_event_disable+0x26/0x80
> [   34.366175] Code: 90 90 90 90 55 48 89 fd 53 48 8b 47 10 8b 90 38
> 01 00 00 85 d2 75 13 48 8b 88 40 01 00 00 48 8d 90 40
> [   34.367340] RSP: 0018:ffffc90001e5bdd0 EFLAGS: 00010287
> [   34.367651] RAX: ffff88810d86cc00 RBX: ffff88810d86cce0 RCX: ffff8881075c8168
> [   34.368067] RDX: ffff88810d86cd40 RSI: 0000000000000003 RDI: ffff88810a5e1db0
> [   34.368482] RBP: ffff88810a5e1db0 R08: 00000000ffffffff R09: 0000000000000000
> [   34.368896] R10: 0000000000000001 R11: 0000000000000000 R12: ffff8881075c8250
> [   34.369310] R13: ffff8881075c82f0 R14: ffffc90001e5bb80 R15: ffff8881075c8000
> [   34.369775]  trace_uprobe_register+0x1a8/0x300
> [   34.370053]  perf_trace_event_unreg.isra.0+0x22/0x80
> [   34.370342]  perf_uprobe_destroy+0x3a/0x70
> [   34.370582]  _free_event+0x114/0x580
> [   34.370806]  perf_event_release_kernel+0x282/0x2c0
> [   34.371123]  perf_release+0x11/0x20
> [   34.371354]  __fput+0x102/0x2e0
> [   34.371561]  task_work_run+0x55/0xa0
> [   34.371798]  syscall_exit_to_user_mode+0x1dd/0x1f0
> [   34.372104]  do_syscall_64+0x70/0x140
> [   34.372337]  entry_SYSCALL_64_after_hwframe+0x76/0x7e
> [   34.372659] RIP: 0033:0x7fd71c986a94
> [   34.372901] Code: 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84
> 00 00 00 00 00 90 f3 0f 1e fa 80 3d d5 18 0e 00 00 73
> [   34.373987] RSP: 002b:00007ffc09d83118 EFLAGS: 00000202 ORIG_RAX:
> 0000000000000003
> [   34.374426] RAX: 0000000000000000 RBX: 00007ffc09d835e8 RCX: 00007fd71c986a94
> [   34.374874] RDX: 000000000000000b RSI: 0000000000002401 RDI: 000000000000000c
> [   34.375325] RBP: 00007ffc09d83150 R08: 00000000049d39c7 R09: 0000000000000007
> [   34.375782] R10: 00000000049d71b0 R11: 0000000000000202 R12: 0000000000000000
> [   34.376245] R13: 00007ffc09d83608 R14: 00007fd71cae3000 R15: 000000000103cd90
> [   34.376661]  </TASK>
> [   34.376794] Modules linked in: bpf_testmod(OE) aesni_intel(E)
> crypto_simd(E) cryptd(E) kvm_intel(E) i2c_piix4(E) i2c_s)
> [   34.377900] CR2: ffffc90001c5bea0
> [   34.378097] ---[ end trace 0000000000000000 ]---
> [   34.378366] RIP: 0010:__wake_up_common+0x3e/0xc0
> [   34.378637] Code: 89 d4 55 53 48 89 fb 48 83 ec 08 8b 05 c3 05 a8
> 02 89 74 24 04 85 c0 75 6d 48 8b 43 48 48 83 c3 48 4f
> [   34.379703] RSP: 0018:ffffc900001d0d90 EFLAGS: 00010087
> [   34.380004] RAX: ffffc90001c5bea0 RBX: ffff88810138e0e8 RCX: 0000000000000001
> [   34.380414] RDX: 0000000080010005 RSI: ffffffff8295f82c RDI: ffffc90001c5be88
> [   34.380827] RBP: ffff88810138e0a0 R08: 0000000000000000 R09: 000000000000438f
> [   34.381284] R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000001
> [   34.381736] R13: 0000000000000046 R14: 0000000000000000 R15: 0000000000000000
> [   34.382198] FS:  00007fd71c7b3d00(0000) GS:ffff88881ca00000(0000)
> knlGS:0000000000000000
> [   34.382712] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [   34.383082] CR2: ffffc90001c5bea0 CR3: 000000010b456005 CR4: 0000000000370ef0
> [   34.383540] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [   34.383991] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [   34.384407] Kernel panic - not syncing: Fatal exception in interrupt
> [   35.464708] Shutting down cpus with NMI
> [   35.465244] Kernel Offset: disabled
> [   35.465474] ---[ end Kernel panic - not syncing: Fatal exception in
> interrupt ]---

Powered by blists - more mailing lists