linux-kernel - Re: [PATCH] printk: do cond_resched() between lines while outputting to consoles

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Thu, 26 Nov 2015 09:53:45 +0100
From:	Jan Kara <jack@...e.cz>
To:	Tejun Heo <tj@...nel.org>
Cc:	Jan Kara <jack@...e.cz>, Andrew Morton <akpm@...ux-foundation.org>,
	Calvin Owens <calvinowens@...com>,
	Dave Jones <davej@...emonkey.org.uk>,
	Kyle McMartin <kyle@...nel.org>, linux-kernel@...r.kernel.org,
	kernel-team@...com
Subject: Re: [PATCH] printk: do cond_resched() between lines while outputting
 to consoles

  Hello,

On Wed 25-11-15 12:02:17, Tejun Heo wrote:
> On Wed, Nov 25, 2015 at 10:05:22AM +0100, Jan Kara wrote:
> > So did you particularly have an issue during console registration? Because
> 
> Yeap, we're seeing a small ratio of machines falling head over hills
> during IPMI serial console registration.  Pumping out the messages
> collected prior to registration takes too long triggering softlockup
> warning on all forty something CPUs which pile a metric ton of
> messages atop.  From then on, softlockup / rcu stall warnings repeat
> themselves.  Some machines recover after >10mins of doing that.  The
> log is hillarious to look at afterward.

OK, then feel free to add my:

Acked-by: Jan Kara <jack@...e.com>

> > at least our customers mostly have issues during heavy use of ordinary
> > printk (e.g. during boot or when hardware gets probed) and your change
> > doesn't affect that case. That being said if you really hit a case where
> 
> Hah, that must be a lot of messages being printk'd.

Yes, it is. They have ~1000 SCSI devices attached (250 disks, each over 4
paths) and similar stuff. But also doing sysrq-t on a large machine
generates enough output to kill the machine...

> > your patch helps, then I have no problem with it (you can add my Acked-by).
> > 
> > At Kernel Summit I spoke with Linus and Andrew regarding printk softlockups
> > and we ended up with a decision that we decouple queueing into kernel
> > ringbuffer from the actual printing into console which would happen from
> > kthread / workqueue. Then the lockups would be solved by printing to
> > console happening from schedulable context and printk() as such being
> > independent from console speed. We only have to have some special cases
> > there for crashes so that messages get printed synchronously in that case.
> 
> Yeah, we'd prolly want to make the behavior contingent on the time
> taken and so on.  At any rate, even with workqueue-deferred dumping,
> this patch would still be necessary for non-preemptible kernels;
> otherwise, there's no cond_resched() in printing path right now.

Yup.

								Honza
-- 
Jan Kara <jack@...e.com>
SUSE Labs, CR
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/