linux-kernel - Re: [RFC PATCH] trace/perf: cure locking issue in perf_event

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170428142735.tyhp6xwljqpdbms2@linutronix.de>
Date:   Fri, 28 Apr 2017 16:27:35 +0200
From:   Sebastian Siewior <bigeasy@...utronix.de>
To:     Thomas Gleixner <tglx@...utronix.de>,
        Peter Zijlstra <peterz@...radead.org>
Cc:     LKML <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...nel.org>,
        Steven Rostedt <rostedt@...dmis.org>
Subject: Re: [RFC PATCH] trace/perf: cure locking issue in perf_event_open()
 error path

With the last patch on-top I trigger this now and then:

 ======================================================
 WARNING: possible circular locking dependency detected
 4.11.0-rc8-00894-g8bd462ee4aac-dirty #84 Not tainted
 ------------------------------------------------------
 trinity-subchil/4966 is trying to acquire lock:
  (cpu_hotplug_lock.rw_sem){++++++}, at: [<ffffffff812fcdad>] tp_perf_event_destroy+0xd/0x20

 but task is already holding lock:
  (&ctx->mutex){+.+.+.}, at: [<ffffffff81316ede>] perf_event_exit_task+0x2ae/0x8d0

 which lock already depends on the new lock.


 the existing dependency chain (in reverse order) is:

 -> #2 (&ctx->mutex){+.+.+.}:
        __lock_acquire+0x2534/0x2cd0
        lock_acquire+0x27c/0x2f0
        __mutex_lock+0xef/0x1280
        mutex_lock_nested+0x16/0x20
        SyS_perf_event_open+0x11ab/0x1e80
        entry_SYSCALL_64_fastpath+0x23/0xc2

 -> #1 (&sig->cred_guard_mutex){+.+.+.}:
        __lock_acquire+0x2534/0x2cd0
        lock_acquire+0x27c/0x2f0
        __mutex_lock+0xef/0x1280
        mutex_lock_interruptible_nested+0x16/0x20
        SyS_perf_event_open+0x1bd6/0x1e80
        entry_SYSCALL_64_fastpath+0x23/0xc2

 -> #0 (cpu_hotplug_lock.rw_sem){++++++}:
        check_prevs_add+0x544/0x18f0
        __lock_acquire+0x2534/0x2cd0
        lock_acquire+0x27c/0x2f0
        get_online_cpus+0x3d/0x80
        tp_perf_event_destroy+0xd/0x20
        _free_event+0x61b/0x800
        free_event+0x68/0x70
        perf_event_exit_task+0x816/0x8d0
        do_exit+0xc90/0x2a90
        do_group_exit+0x1aa/0x2b0
        SyS_exit_group+0x18/0x20
        entry_SYSCALL_64_fastpath+0x23/0xc2

 other info that might help us debug this:

 Chain exists of:
   cpu_hotplug_lock.rw_sem --> &sig->cred_guard_mutex --> &ctx->mutex

  Possible unsafe locking scenario:

        CPU0                    CPU1
        ----                    ----
   lock(&ctx->mutex);
                                lock(&sig->cred_guard_mutex);
                                lock(&ctx->mutex);
   lock(cpu_hotplug_lock.rw_sem);

  *** DEADLOCK ***

 1 lock held by trinity-subchil/4966:
  #0:  (&ctx->mutex){+.+.+.}, at: [<ffffffff81316ede>] perf_event_exit_task+0x2ae/0x8d0

 stack backtrace:
 CPU: 3 PID: 4966 Comm: trinity-subchil Not tainted 4.11.0-rc8-00894-g8bd462ee4aac-dirty #84
 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 1.10.2-1 04/01/2014
 Call Trace:
  dump_stack+0x86/0xce
  print_circular_bug+0x5c3/0x620
  ? lockdep_on+0x50/0x50
  check_prevs_add+0x544/0x18f0
  ? check_irq_usage+0x150/0x150
  ? lock_acquire+0x27c/0x2f0
  __lock_acquire+0x2534/0x2cd0
  ? __lock_acquire+0x2534/0x2cd0
  ? find_held_lock+0x36/0x1c0
  lock_acquire+0x27c/0x2f0
  ? tp_perf_event_destroy+0xd/0x20
  get_online_cpus+0x3d/0x80
  ? tp_perf_event_destroy+0xd/0x20
  tp_perf_event_destroy+0xd/0x20
  _free_event+0x61b/0x800
  free_event+0x68/0x70
  perf_event_exit_task+0x816/0x8d0
  do_exit+0xc90/0x2a90
  ? mm_update_next_owner+0x550/0x550
  ? getname_flags+0xde/0x370
  ? getname_flags+0xde/0x370
  ? rcu_read_lock_sched_held+0x14a/0x180
  ? prepare_bprm_creds+0xf0/0xf0
  ? kmem_cache_free+0x250/0x2c0
  ? getname_flags+0xde/0x370
  ? entry_SYSCALL_64_fastpath+0x5/0xc2
  do_group_exit+0x1aa/0x2b0
  SyS_exit_group+0x18/0x20
  entry_SYSCALL_64_fastpath+0x23/0xc2

and I am not sure what to do here…

Sebastian