linux-kernel - [PATCH 3.19.y-ckt 122/210] panic: release stale console lock to always get the logbuf printed out

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1454030348-17736-123-git-send-email-kamal@canonical.com>
Date:	Thu, 28 Jan 2016 17:17:40 -0800
From:	Kamal Mostafa <kamal@...onical.com>
To:	linux-kernel@...r.kernel.org, stable@...r.kernel.org,
	kernel-team@...ts.ubuntu.com
Cc:	Vitaly Kuznetsov <vkuznets@...hat.com>,
	HATAYAMA Daisuke <d.hatayama@...fujitsu.com>,
	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>,
	Jiri Kosina <jkosina@...e.cz>, Baoquan He <bhe@...hat.com>,
	Prarit Bhargava <prarit@...hat.com>,
	Xie XiuQi <xiexiuqi@...wei.com>,
	Seth Jennings <sjenning@...hat.com>,
	"K. Y. Srinivasan" <kys@...rosoft.com>, Jan Kara <jack@...e.cz>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Kamal Mostafa <kamal@...onical.com>
Subject: [PATCH 3.19.y-ckt 122/210] panic: release stale console lock to always get the logbuf printed out

3.19.8-ckt14 -stable review patch.  If anyone has any objections, please let me know.

---8<------------------------------------------------------------

From: Vitaly Kuznetsov <vkuznets@...hat.com>

commit 08d78658f393fefaa2e6507ea052c6f8ef4002a2 upstream.

In some cases we may end up killing the CPU holding the console lock
while still having valuable data in logbuf. E.g. I'm observing the
following:

- A crash is happening on one CPU and console_unlock() is being called on
  some other.

- console_unlock() tries to print out the buffer before releasing the lock
  and on slow console it takes time.

- in the meanwhile crashing CPU does lots of printk()-s with valuable data
  (which go to the logbuf) and sends IPIs to all other CPUs.

- console_unlock() finishes printing previous chunk and enables interrupts
  before trying to print out the rest, the CPU catches the IPI and never
  releases console lock.

This is not the only possible case: in VT/fb subsystems we have many other
console_lock()/console_unlock() users.  Non-masked interrupts (or
receiving NMI in case of extreme slowness) will have the same result.
Getting the whole console buffer printed out on crash should be top
priority.

[akpm@...ux-foundation.org: tweak comment text]
Signed-off-by: Vitaly Kuznetsov <vkuznets@...hat.com>
Cc: HATAYAMA Daisuke <d.hatayama@...fujitsu.com>
Cc: Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>
Cc: Jiri Kosina <jkosina@...e.cz>
Cc: Baoquan He <bhe@...hat.com>
Cc: Prarit Bhargava <prarit@...hat.com>
Cc: Xie XiuQi <xiexiuqi@...wei.com>
Cc: Seth Jennings <sjenning@...hat.com>
Cc: "K. Y. Srinivasan" <kys@...rosoft.com>
Cc: Jan Kara <jack@...e.cz>
Signed-off-by: Andrew Morton <akpm@...ux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@...ux-foundation.org>

Signed-off-by: Kamal Mostafa <kamal@...onical.com>
---
 kernel/panic.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/kernel/panic.c b/kernel/panic.c
index 4d8d6f9..1871216 100644
--- a/kernel/panic.c
+++ b/kernel/panic.c
@@ -23,6 +23,7 @@
 #include <linux/sysrq.h>
 #include <linux/init.h>
 #include <linux/nmi.h>
+#include <linux/console.h>

 #define PANIC_TIMER_STEP 100
 #define PANIC_BLINK_SPD 18
@@ -146,6 +147,15 @@ void panic(const char *fmt, ...)

 	bust_spinlocks(0);

+	/*
+	 * We may have ended up stopping the CPU holding the lock (in
+	 * smp_send_stop()) while still having some valuable data in the console
+	 * buffer.  Try to acquire the lock then release it regardless of the
+	 * result.  The release will also print the buffers out.
+	 */
+	console_trylock();
+	console_unlock();
+
 	if (!panic_blink)
 		panic_blink = no_blink;

-- 
1.9.1