linux-kernel - Re: printk deadlock due to double lock attempt on current CPU's runqueue

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 10 Nov 2021 11:44:14 +0100
From:   Daniel Vetter <daniel@...ll.ch>
To:     John Ogness <john.ogness@...utronix.de>
Cc:     Daniel Vetter <daniel@...ll.ch>,
        Peter Zijlstra <peterz@...radead.org>,
        Sultan Alsawaf <sultan@...neltoast.com>,
        Anton Vorontsov <anton@...msg.org>,
        Ben Segall <bsegall@...gle.com>,
        Colin Cross <ccross@...roid.com>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        David Airlie <airlied@...ux.ie>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        dri-devel@...ts.freedesktop.org, Ingo Molnar <mingo@...hat.com>,
        Juri Lelli <juri.lelli@...hat.com>,
        Kees Cook <keescook@...omium.org>,
        linux-kernel@...r.kernel.org,
        Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>,
        Maxime Ripard <mripard@...nel.org>,
        Mel Gorman <mgorman@...e.de>, Petr Mladek <pmladek@...e.com>,
        Sergey Senozhatsky <senozhatsky@...omium.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Thomas Zimmermann <tzimmermann@...e.de>,
        Tony Luck <tony.luck@...el.com>,
        Vincent Guittot <vincent.guittot@...aro.org>, mkoutny@...e.com
Subject: Re: printk deadlock due to double lock attempt on current CPU's
 runqueue

On Wed, Nov 10, 2021 at 11:13:37AM +0106, John Ogness wrote:
> On 2021-11-10, Daniel Vetter <daniel@...ll.ch> wrote:
> > I'm a bit out of the loop but from lwn articles my understanding is
> > that part of upstreaming from -rt we no longer have the explicit "I'm
> > a safe console for direct printing" opt-in. Which I get from a
> > backwards compat pov, but I still think for at least fbcon we really
> > should never attempt a direct printk con->write, it's just all around
> > terrible.
> 
> Right now we don't have an explicit "I'm a safe console for direct
> printing" option. Right now all printing is direct. But it sounds to me
> that we should add this console flag when we introduce kthread printers.

Ah I wasnt' clear whether this was merged already or not yet.

> > So yeah for fbcon at least I think we really should throw out direct
> > con->write from printk completely.
> 
> Even after we introduce kthread printers, there will still be situations
> where direct printing is used: booting (before kthreads exist) and
> shutdown/suspend/crash situations, when the kthreads may not be
> active.
> 
> I will introduce a console flag so that consoles can opt-out for direct
> printing. (opt-out rather than opt-in is probably easier, since there
> are only a few that would need to opt-out).

Yeah opt-out for fbcon sounds good to me, and then across the board (even
when the kthread is not yet or no longer running, it really doesn't make
anything better). Maybe with a fbcon module (or system wide
force_direct_printk) option to disable that flag in case someone needs
it.

> Since kthread printers do not yet exist (hoping to get them in for
> 5.17), I am not sure how we should address the reported bug for existing
> kernels.

I think we just can't. This thing keeps popping up in all kinds of places,
and in some cases it's really not possible to fix it. Like in
oops_in_progress we can just give up, but for normal printk from funny
places we'd corrupt the console (plus there's no flag to even detect these
cases), since we really need to launch these work_struct to get stuff to
the screen.

I think in the past we sometimes managed to work around the issue by
moving the printk outside of locks, but that's rarely a good idea just
because fbcon.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch