linux-kernel - Deadlock in fb and tty

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 11 Sep 2013 10:25:53 +0100
From:	John Tapsell <johnflux@...il.com>
To:	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Deadlock in fb and tty

Hi,
  I'm consistently and constantly hitting a deadlock.

console_callback in drivers/tty/vt/vt.c does:  console_lock()  and then calls:
  do_blank_screen, which calls:
    vc->vc_sw->con_blank(..)  which can be a pointer to the function:
      fbcon_blank in video/console/fbcon.c.  This is missing a
WARN_CONSOLE_UNLOCKED.  This calls:
        fbcon_generic_blank which does lock_fb_info(info)

So we have a console_lock() followed by a lock_fb_info(info).

Now if while that is running, we have an ioctl call:

do_fb_ioctl in drivers/video/fbmem.c which does:
               if (!lock_fb_info(info))
                        return -ENODEV;
                console_lock();


So it tries to acquire the same locks in the reverse order.  This
deadlocks consistently for me.

(I'm also curious why I'm hitting this continually, when everyone else
seems to be okay.)

I understand that this is really difficult to fix, but if anyone has
even a suggestion on how to hack it to make it work for me, I'd be
very grateful.

Thank you,

John Tapsell
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/