linux-kernel - Re: [RFC] softlockup: serialized softlockup's log

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAD=FV=UJc3Ln7KDXVfXfpmbWmmbp2v=1o4T_11nv2R=bWttZeQ@mail.gmail.com>
Date:   Mon, 27 Nov 2023 13:55:16 -0800
From:   Doug Anderson <dianders@...omium.org>
To:     Petr Mladek <pmladek@...e.com>
Cc:     lizhe.67@...edance.com, akpm@...ux-foundation.org,
        kernelfans@...il.com, lecopzer.chen@...iatek.com,
        linux-kernel@...r.kernel.org, lizefan.x@...edance.com,
        John Ogness <john.ogness@...utronix.de>
Subject: Re: [RFC] softlockup: serialized softlockup's log

Hi,

On Mon, Nov 27, 2023 at 9:13 AM Petr Mladek <pmladek@...e.com> wrote:
>
> > I've tried several times, but unfortunately I haven't been able to
> > reproduce the problem you mentioned. My concern is that if the lock
> > is shared, there will be potential deadlock issues because hardlockup
> > exploits nmi.
>
> Yes, one has to be very careful when using locks under NMI.
> I am aware of three approaches:
>
>   1. It should be safe to use a lock which is used only in NMI.
>      The only problem might be ABBA deadlocks.
>
>      But this is not usable to serialize softlockup and hardlockup.
>
>
>   2. Using try_lock() in NMI and skipping the report when the lock
>      could not be taken.
>
>     But we probably should not skip the detailed info printed by the
>     hardlockup detector just because another softlockup or
>     hardlockup report is running in parallel.
>
>
>   3. Using a lock which might be nested on the same CPU.
>
>     It does not prevent mixed output completely. The nested
>     context will print the messages in the middle of the outer
>     one on the same CPU. But it would serialize output against
>     other CPUs.
>
>     There already is one used to serialize some backtraces,
>     see printk_cpu_sync_get_irqsave() in dump_stack_lvl() and
>     nmi_cpu_backtrace().
>
>     It should be safe when the locked code does not wait for
>     other CPUs. And when it is a leaf lock so that there
>     is no need to think about ABBA deadlocks against
>     other locks.
>
>     IMHO, printk_cpu_sync_get_irqsave() probably should be
>     safe to serialize the softlockup and hardlockup
>     detectors.
>
>
> Adding John into Cc. He is familiar with
> printk_cpu_sync_get_irqsave(). He wanted to use it also
> for synchronizing console drivers. Well, it will be done
> another way in the end.

Thanks for all the detailed info! Using printk_cpu_sync_get_irqsave()
looks pretty promising. I'll try to do some testing with this and then
aim to send a patch later this week that builds upon Li Zhe's patch.
If everything works out I'll probably remove the newly added spinlock
and use printk_cpu_sync_get_irqsave() instead, but the spinlock should
work fine in the short term in any case.

-Doug