linux-kernel - Re: v5.9-rc3-rt3 boot time networking lockdep splat

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 8 Sep 2020 14:19:02 +0200
From:   Sebastian Andrzej Siewior <bigeasy@...utronix.de>
To:     Mike Galbraith <efault@....de>
Cc:     Thomas Gleixner <tglx@...utronix.de>,
        LKML <linux-kernel@...r.kernel.org>,
        linux-rt-users <linux-rt-users@...r.kernel.org>,
        Steven Rostedt <rostedt@...dmis.org>
Subject: Re: v5.9-rc3-rt3 boot time networking lockdep splat

On 2020-09-05 06:47:29 [+0200], Mike Galbraith wrote:
> [   22.024936] ======================================================
> [   22.024936] WARNING: possible circular locking dependency detected
> [   22.024937] 5.9.0.gc70672d-rt3-rt #8 Tainted: G            E
> [   22.024938] ------------------------------------------------------
> [   22.024939] ksoftirqd/0/10 is trying to acquire lock:
> [   22.024941] ffff983475521278 (&sch->q.lock){+...}-{0:0}, at: sch_direct_xmit+0x81/0x2f0
> [   22.024947]
>                but task is already holding lock:
> [   22.024947] ffff9834755212b8 (&s->seqcount#9){+...}-{0:0}, at: br_dev_queue_push_xmit+0x7d/0x180 [bridge]
> [   22.024959]
>                which lock already depends on the new lock.
> 
> [   22.024960]
>                the existing dependency chain (in reverse order) is:
> [   22.024961]
>                -> #1 (&s->seqcount#9){+...}-{0:0}:
> [   22.024963]        lock_acquire+0x92/0x3f0
> [   22.024967]        __dev_queue_xmit+0xce7/0xe30
…
>                -> #0 (&sch->q.lock){+...}-{0:0}:
> [   22.025015]        validate_chain+0xa81/0x1230
> [   22.025016]        __lock_acquire+0x880/0xbf0
> [   22.025017]        lock_acquire+0x92/0x3f0
> [   22.025018]        rt_spin_lock+0x78/0xd0
> [   22.025020]        sch_direct_xmit+0x81/0x2f0
…

>                other info that might help us debug this:
> 
> [   22.025061]  Possible unsafe locking scenario:
> 
> [   22.025061]        CPU0                    CPU1
> [   22.025061]        ----                    ----
> [   22.025062]   lock(&s->seqcount#9);
> [   22.025064]                                lock(&sch->q.lock);
> [   22.025065]                                lock(&s->seqcount#9);
> [   22.025065]   lock(&sch->q.lock);
> [   22.025066]
>                 *** DEADLOCK ***

This has nothing to do with the bridge but with the fact that you use a
non standard queue class (something else than pfifo_fast).

The flow in CPU1 is the default flow but the second lock is a trylock.
CPU0 is from sch_direct_xmit() where it drops the the
root_lock/qdisc.lock and re-acquires it. This shouldn't fail because the
CPU1 a try-lock of the seqlock first and then the seqcount is "not
acquired". So if we annotate the seqcount as a try_acquire then it
should not do this anymore.

Sebastian