linux-kernel - [BUG] kernel freeze, rcu_sched self-detected stall on CPU

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Sat, 17 Dec 2016 23:25:35 +0100
From:   Johanna Abrahamsson <johanna@...o.org>
To:     Andrew Morton <akpm@...ux-foundation.org>,
        Davidlohr Bueso <dave@...olabs.net>
Cc:     linux-kernel@...r.kernel.org
Subject: [BUG] kernel freeze, rcu_sched self-detected stall on CPU

A bug has been introduced in ipc/sem.c by a patch '[PATCH -next 2/2] ipc/sem: avoid idr tree lookup for interrupted semop' (https://marc.info/?l=linux-kernel&m=147870885903619, commit 370b262c896e5565b271a3ea3abee4d0914ba443).
The bug results in a kernel freeze and several "INFO: rcu_sched self-detected stall on CPU". The system becomes unresponsible and has to be rebooted to work again as far as I can tell.
My repro case is, hilariously enough, packaging the linux kernel for debian with `make -j8 deb-pkg`. I'm sure a smaller repro case could be constructed, but so far I've had 100% reproducibility from this one. The freeze always occurs at the Makefiles.headerinst part of packaging, after the kernel is built.
The bug is present in next-20161124 and linux-next versions going forward from that and has now been merged to linus/master. However, in linus/master it doesn't trigger a bug. This seems to indicate that it only triggers a bug in combination with a commit that is present in next-20161216 but not present in linus/master.
I've confirmed that doing `git revert 370b262c896e5565b271a3ea3abee4d0914ba443` on next-20161216 fixes the bug.

In all repro cases I've tested on virtual machines with multiple cpu cores running under either Xen or Virtualbox.

I will try to investigate this further but as I have limited knowledge of RCU and how the kernel works with semaphores don't expect any miracles :)

Best Regards,
Johanna Abrahamsson