lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Date:	Fri, 11 Dec 2015 16:16:37 -0800 (PST)
From:	Ani Sinha <ani@...sta.com>
To:	"Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>
cc:	Rik van Riel <riel@...hat.com>, Ani Sinha <ani@...sta.com>,
	Randy Dunlap <rdunlap@...radead.org>,
	Richard Weinberger <richard@....at>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	Ivan Delalande <colona@...sta.com>,
	fruggeri <fruggeri@...sta.com>
Subject: Re: new warning on sysrq kernel crash trigger



On Fri, 11 Dec 2015, Paul E. McKenney wrote:

> On Fri, Dec 11, 2015 at 05:10:43PM -0500, Rik van Riel wrote:
> > On 12/11/2015 03:44 PM, Ani Sinha wrote:
> > > 
> > > 
> > > On Thu, 10 Dec 2015, Paul E. McKenney wrote:
> > > 
> > >> On Thu, Dec 10, 2015 at 03:57:09PM -0800, Ani Sinha wrote:
> > >>> Hi guys
> > >>>
> > >>> I am noticing a new warning in linux 3.18 which we did not see before
> > >>> in linux 3.4 :
> > >>>
> > >>> bash-4.1# echo c > /proc/sysrq-trigger
> > >>> [  978.807185] BUG: sleeping function called from invalid context at
> > >>> ../arch/x86/mm/fault.c:1187
> > >>> [  978.909816] in_atomic(): 0, irqs_disabled(): 0, pid: 4706, name: bash
> > >>> [  978.987358] Preemption disabled at:[<ffffffff81484339>] printk+0x48/0x4a
> > >>>
> > >>>
> > >>> I have bisected this to the following change :
> > >>>
> > >>> commit 984d74a72076a12b400339973e8c98fd2fcd90e5
> > >>> Author: Rik van Riel <riel@...hat.com>
> > >>> Date:   Fri Jun 6 14:38:13 2014 -0700
> > >>>
> > >>>     sysrq: rcu-ify __handle_sysrq
> > >>>
> > >>>
> > >>> the rcu_read_lock() in handle_sysrq() bumps up
> > >>> current->rcu_read_lock_nesting. Hence, in __do_page_fault() when it
> > >>> calls might_sleep() in x86/mm/fault.c line 1191,
> > >>> preempt_count_equals(0) returns false and hence the warning is
> > >>> printed.
> > >>>
> > >>> One way to handle this would be to do something like this:
> > >>>
> > >>> diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c
> > >>> index eef44d9..d4dbe22 100644
> > >>> --- a/arch/x86/mm/fault.c
> > >>> +++ b/arch/x86/mm/fault.c
> > >>> @@ -1132,7 +1132,7 @@ __do_page_fault(struct pt_regs *regs, unsigned
> > >>> long error_code,
> > >>>   * If we're in an interrupt, have no user context or are running
> > >>>   * in a region with pagefaults disabled then we must not take the fault
> > >>>   */
> > >>> - if (unlikely(faulthandler_disabled() || !mm)) {
> > >>> + if (unlikely(faulthandler_disabled() || rcu_preempt_depth() || !mm)) {
> > >>
> > >> This works if CONFIG_PREEMPT=y, but if CONFIG_PREEMPT=n, then
> > >> rcu_preempt_depth() unconditionally returns zero.  And if
> > >> CONFIG_PREEMPT_COUNT=y && CONFIG_PREEMPT=n, you would still see
> > >> the might_sleep() splat.
> > >>
> > >> Maybe use SRCU instead of RCU for this purpose?
> > >>
> > > 
> > > From ae232ce3fb167b2ad363bfac7aab69001bc55a50 Mon Sep 17 00:00:00 2001
> > > From: Ani Sinha <ani@...sta.com>
> > > Date: Fri, 11 Dec 2015 12:07:42 -0800
> > > Subject: [PATCH 1/1] Fix 'sleeping function called from invalid context'
> > >  warning in sysrq generated crash.
> > > 
> > > Commit 984d74a72076a1 ("sysrq: rcu-ify __handle_sysrq") 
> > > replaced spin_lock_irqsave() calls with
> > > rcu_read_lock() calls in sysrq. Since rcu_read_lock() does not
> > > disable preemption, faulthandler_disabled() in
> > > __do_page_fault() in x86/fault.c returns false. When the code
> > > later calls might_sleep() in the pagefault handler, we get the
> > > following warning:
> > > 
> > > BUG: sleeping function called from invalid context at ../arch/x86/mm/fault.c:1187
> > > in_atomic(): 0, irqs_disabled(): 0, pid: 4706, name: bash
> > > Preemption disabled at:[<ffffffff81484339>] printk+0x48/0x4a
> > > 
> > > To fix this, replace RCU call in handle_sysrq() to use SRCU.
> > 
> > The sysrq code can be called from irq context.
> > 
> > Trying to use SRCU from an irq context sounds like it could
> > be a bad idea, though admittedly I do not know enough about
> > SRCU to know for sure :)
> 
> Indeed, not the best idea!  ;-)
> 
> I could imagine something like this:
> 
> 	if (in_irq())
> 		rcu_read_lock();
> 	else
> 		idx = srcu_read_lock(&sysrq_rcu);
> 
> And ditto for unlock.  Then, for the update:
> 
> 	synchronize_rcu_mult(call_rcu, call_sysrq_srcu);
> 
> Where:
> 
> 	static void call_sysrq_srcu(struct rcu_head *head, rcu_callback_t func)
> 	{
> 		call_srcu(&sysrq_rcu, head, func);
> 	}
> 

>From ae232ce3fb167b2ad363bfac7aab69001bc55a50 Mon Sep 17 00:00:00 2001
From: Ani Sinha <ani@...sta.com>
Date: Fri, 11 Dec 2015 12:07:42 -0800
Subject: [PATCH 1/1] Fix 'sleeping function called from invalid context'
 warning in sysrq generated crash.

Commit 984d74a72076a1 ("sysrq: rcu-ify __handle_sysrq") 
replaced spin_lock_irqsave() calls with
rcu_read_lock() calls in sysrq. Since rcu_read_lock() does not
disable preemption, faulthandler_disabled() in
__do_page_fault() in x86/fault.c returns false. When the code
later calls might_sleep() in the pagefault handler, we get the
following warning:

BUG: sleeping function called from invalid context at ../arch/x86/mm/fault.c:1187
in_atomic(): 0, irqs_disabled(): 0, pid: 4706, name: bash
Preemption disabled at:[<ffffffff81484339>] printk+0x48/0x4a

To fix this, replace RCU call in handle_sysrq() to use SRCU
in non-irq context.

Tested this patch on linux 3.18 by booting off one of our boards.

Fixes: 984d74a72076a1 ("sysrq: rcu-ify __handle_sysrq")

Signed-off-by: Ani Sinha <ani@...sta.com>
---
diff --git a/drivers/tty/sysrq.c b/drivers/tty/sysrq.c
index 5381a72..df7d747 100644
--- a/drivers/tty/sysrq.c
+++ b/drivers/tty/sysrq.c
@@ -54,6 +54,7 @@
 /* Whether we react on sysrq keys or just ignore them */
 static int __read_mostly sysrq_enabled = CONFIG_MAGIC_SYSRQ_DEFAULT_ENABLE;
 static bool __read_mostly sysrq_always_enabled;
+DEFINE_STATIC_SRCU(sysrq_rcu);
 
 static bool sysrq_on(void)
 {
@@ -519,10 +520,13 @@ void __handle_sysrq(int key, bool check_mask)
 {
 	struct sysrq_key_op *op_p;
 	int orig_log_level;
-	int i;
+	int i, idx;
 
 	rcu_sysrq_start();
-	rcu_read_lock();
+	if (in_irq())
+		rcu_read_lock();
+	else
+		idx = srcu_read_lock(&sysrq_rcu);
 	/*
 	 * Raise the apparent loglevel to maximum so that the sysrq header
 	 * is shown to provide the user with positive feedback.  We do not
@@ -564,7 +568,10 @@ void __handle_sysrq(int key, bool check_mask)
 		pr_cont("\n");
 		console_loglevel = orig_log_level;
 	}
-	rcu_read_unlock();
+	if (in_irq())
+		rcu_read_unlock();
+	else
+		srcu_read_unlock(&sysrq_rcu, idx);
 	rcu_sysrq_end();
 }
 
@@ -1040,6 +1047,11 @@ int sysrq_toggle_support(int enable_mask)
 	return 0;
 }
 
+static void call_sysrq_srcu(struct rcu_head *head, rcu_callback_t func)
+{
+	call_srcu(&sysrq_rcu, head, func);
+}
+
 static int __sysrq_swap_key_ops(int key, struct sysrq_key_op *insert_op_p,
                                 struct sysrq_key_op *remove_op_p)
 {
@@ -1059,7 +1071,7 @@ static int __sysrq_swap_key_ops(int key, struct sysrq_key_op *insert_op_p,
 	 * Wait for it to go away before returning, so the code for an old
 	 * op is not freed (eg. on module unload) while it is in use.
 	 */
-	synchronize_rcu();
+	synchronize_rcu_mult(call_rcu, call_sysrq_srcu);
 
 	return retval;
 }
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists