linux-kernel - Re: [4.9-rc3] BUG: unable to handle kernel paging request at ffffc900144dfc60

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+55aFzphURPFzAvU4z6Moy7ZmimcwPuUdYU8bj9z0J+S8X1rw@mail.gmail.com>
Date:   Tue, 1 Nov 2016 17:47:18 -0600
From:   Linus Torvalds <torvalds@...ux-foundation.org>
To:     Tetsuo Handa <penguin-kernel@...ove.sakura.ne.jp>,
        Peter Zijlstra <peterz@...radead.org>,
        Ingo Molnar <mingo@...hat.com>
Cc:     Andrew Lutomirski <luto@...nel.org>,
        "the arch/x86 maintainers" <x86@...nel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        Brian Gerst <brgerst@...il.com>,
        Borislav Petkov <bp@...en8.de>, Jann Horn <jann@...jh.net>,
        Linux API <linux-api@...r.kernel.org>,
        Kees Cook <keescook@...omium.org>,
        Tycho Andersen <tycho.andersen@...onical.com>
Subject: Re: [4.9-rc3] BUG: unable to handle kernel paging request at ffffc900144dfc60

On Tue, Nov 1, 2016 at 8:36 AM, Tetsuo Handa
<penguin-kernel@...ove.sakura.ne.jp> wrote:
>
> I got an Oops with khungtaskd. This kernel was built with CONFIG_THREAD_INFO_IN_TASK=y .
> Is this same reason?

CONFIG_THREAD_INFO_IN_TASK is always set on x86, but I assume you also
did VMAP_STACK

And yes, it looks like it's the same "touching another process' stack"
issue, just in sched_show_task() called from check_hung_task(), which
seems to have been due to a watchdog triggering. I'm not sure what the
relationship is with the oom killer happening at the same time, but it
makes the whole thing fairly hard to read.

The cleaned-up oops looks like this:

> [  580.803660] BUG: unable to handle kernel paging request at ffffc900144dfc60
> [  580.807153] IP:  thread_saved_pc+0xb/0x20
> [  580.907040] Call Trace:
> [  580.908547]   sched_show_task+0x50/0x240
> [  580.928793]   watchdog+0x3d0/0x4f0
> [  580.930774]   ? watchdog+0x1fd/0x4f0
> [  580.932785]   ? check_memalloc_stalling_tasks+0x820/0x820
> [  580.935649]   kthread+0xfd/0x120
> [  580.937594]   ? kthread_park+0x60/0x60
> [  580.939693]   ? kthread_park+0x60/0x60
> [  580.941743]   ret_from_fork+0x27/0x40
> [  580.944608] Code: 55 48 8b bf d0 01 00 00 be 00 00 00 02 48 89 e5 e8 6b 58 3f 00 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 8b 87 e0 15 00 00 48 89 e5 <48> 8b 40 30 5d c3 66 66 66 66 66 66 2e 0f 1f 84 00 00 00 00 00
> [  580.952519] RIP  [<ffffffff81026feb>] thread_saved_pc+0xb/0x20
> [  580.954654]  RSP <ffffc900004c3db8>
> [  580.956272] CR2: ffffc900144dfc60

So we have

  watchdog -> check_hung_uninterruptible_task -> check_hung_task ->
sched_show_task -> thread_saved_pc(), which oopses.

We just checked that task was TASK_UNINTERRUPTIBLE in that chain, but
clearly it races with it dying (due to oom), and so by the time er get
to thread_saved_pc() it's dead and the stack is gone.

Considering that we just print out  a useless hex number, not even a
symbol, and there's a big question mark whether this even makes sense
anyway, I suspect we should just remove it all.  The real information
would have come later as part of "show_stack()", which seems to be
doing the proper  try_get_task_stack().

So I _think_ the fix is to just remove this. Perhaps something like
the attached? Adding scheduler people since this is in their code..

              Linus

View attachment "patch.diff" of type "text/plain" (776 bytes)