linux-kernel - Re: printk: preempt_disable with long time resulting in softlockup/RCU stall issues

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20150708081334.GJ32664@pathway.suse.cz>
Date:	Wed, 8 Jul 2015 10:13:34 +0200
From:	Petr Mladek <pmladek@...e.com>
To:	Gavin Hu <gavin.hu.2010@...il.com>
Cc:	linux-kernel@...r.kernel.org, Alex Elder <elder@...aro.org>,
	"Luis R. Rodriguez" <mcgrof@...e.com>,
	"Steven Rostedt (Red Hat)" <rostedt@...dmis.org>,
	Joe Perches <joe@...ches.com>,
	Peter Hurley <peter@...leysoftware.com>,
	Tejun Heo <tj@...nel.org>, cxie4@...vell.com, cldu@...vell.com,
	xjian@...vell.com, fswu@...vell.com, Jan Kara <jack@...e.cz>
Subject: Re: printk: preempt_disable with long time resulting in
 softlockup/RCU stall issues

On Wed 2015-07-08 15:20:01, Gavin Hu wrote:
> Hi,
>      You know, a simple pirntk like printk("hello world!\n") via serial
> console may consume 21 seconds  above, because any printk is obligated to
> pump out all the characters in the log buffer
> with preempt disable until it is empty (vprintk_emit() and
> console_unlock()), not just "hello world!" itself.
> 
> And in the end, long time preempt disable may result in the softlock/RCU
> stall issues. Some systems are configured to trigger a kernel panic when
> meet softlock/ RCU stall.  So heavy printk may result in kernel panic.
> Usually, the solution is to remove related pritnks or limit the printk
> times. So sometimes, the low-level developers have to handle various heavy
> pirntk issues.
> 
> Why don't make the printk mechanism more flexible and configurable, for
> example, just pump out 1000 messages per printk calling,  to avoid  preempt
> disable with a long time?
> And we can disable or enable this feature by a proc interface. Be default,
> this feature is disabled. So it doesn't influence boot-time message printk.
> 
> Anyway, we maybe can give the low-level developers more options.
> 
> Any comments are welcome!

I add Jack into CC because he made several attempts to address this
issue in the past. For example,  see
https://lkml.org/lkml/2013/2/26/743

Jack's approach was similar but also tried to schedule further
writing to the console. With your approach, the console might stay
far behind the reality for a long time.

IMHO, we also want to make sure that all messages go out
when the system is panicking.

Best Regards,
Petr
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/