linux-kernel - Re: printk deadlock due to double lock attempt on current CPU's runqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 11 Nov 2021 10:27:06 +0100
From:   Petr Mladek <pmladek@...e.com>
To:     Peter Zijlstra <peterz@...radead.org>,
        Sultan Alsawaf <sultan@...neltoast.com>,
        Anton Vorontsov <anton@...msg.org>,
        Ben Segall <bsegall@...gle.com>,
        Colin Cross <ccross@...roid.com>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        David Airlie <airlied@...ux.ie>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        dri-devel@...ts.freedesktop.org, Ingo Molnar <mingo@...hat.com>,
        John Ogness <john.ogness@...utronix.de>,
        Juri Lelli <juri.lelli@...hat.com>,
        Kees Cook <keescook@...omium.org>,
        linux-kernel@...r.kernel.org,
        Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>,
        Maxime Ripard <mripard@...nel.org>,
        Mel Gorman <mgorman@...e.de>,
        Sergey Senozhatsky <senozhatsky@...omium.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Thomas Zimmermann <tzimmermann@...e.de>,
        Tony Luck <tony.luck@...el.com>,
        Vincent Guittot <vincent.guittot@...aro.org>, mkoutny@...e.com
Subject: Re: printk deadlock due to double lock attempt on current CPU's
 runqueue

On Wed 2021-11-10 10:37:26, Daniel Vetter wrote:
> On Tue, Nov 09, 2021 at 10:38:47PM +0100, Peter Zijlstra wrote:
> > On Tue, Nov 09, 2021 at 12:06:48PM -0800, Sultan Alsawaf wrote:
> > > Hi,
> > > 
> > > I encountered a printk deadlock on 5.13 which appears to still affect the latest
> > > kernel. The deadlock occurs due to printk being used while having the current
 > > > CPU's runqueue locked, and the underlying framebuffer console attempting to lock
> > > the same runqueue when printk tries to flush the log buffer.
> > 
> > Yes, that's a known 'feature' of some consoles. printk() is in the
> > process of being reworked to not call con->write() from the printk()
> > calling context, which would go a long way towards fixing this.
> 
> And it's getting worse by the year:
> - direct scanout displays (i.e. just a few mmio writes and it will show
>   up) are on the way out at least in laptops, everyone gets self-refresh
>   (dp psr) under software control, so without being able to kick a kthread
>   off nothing shows up except more oopses

printk() tries to avoid this problem by waking processes via
irq_vork(). For example, it is used for waking userspace log
daemons, see wake_up_klogd().

There is one hidden wake_up_process() in up() of @console_sem.
It is "less" critical because it calls the wake_up() only when
there is another process waiting for console_lock(). It must
be a non-printk caller because printk() uses trylock and does
not add waiters. So, there is a non-trivial chance to avoid
the deadlock and the messages.

I wonder if fbdev might use the same trick.

> - because of the impendence mismatch between fbdev and drm-kms we even go
>   ever more this direction for dumb framebuffers, including the firmware
>   boot-up framebuffer simpledrm. This could perhaps be fixed with a new
>   dedicate console driver directly on top of drm-kms, but that's on the
>   wishlist for years and I don't see anyone typing that.
> 
> So yeah for fbcon at least I think we really should throw out direct
> con->write from printk completely.

Yup, that would be great.

Well, I guess that it would be available only for a panic mode when it
will not race with other users?

Best Regards,
Petr