linux-kernel - Re: INFO: task hung in __do_page

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Date:   Thu, 21 Nov 2019 15:00:13 -0600
From:   ebiederm@...ssion.com (Eric W. Biederman)
To:     Dmitry Vyukov <dvyukov@...gle.com>
Cc:     Andy Lutomirski <luto@...nel.org>,
        syzbot <syzbot+6b074f741adbd93d2df5@...kaller.appspotmail.com>,
        Arnaldo Carvalho de Melo <acme@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Arnd Bergmann <arnd@...db.de>,
        Jonathan Corbet <corbet@....net>,
        Kees Cook <keescook@...omium.org>,
        "open list\:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Linux-MM <linux-mm@...ck.org>,
        Dominik Brodowski <linux@...inikbrodowski.net>,
        "Luis R. Rodriguez" <mcgrof@...nel.org>,
        Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        Sudip Mukherjee <sudipm.mukherjee@...il.com>,
        syzkaller-bugs <syzkaller-bugs@...glegroups.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: INFO: task hung in __do_page_fault (2)

Dmitry Vyukov <dvyukov@...gle.com> writes:

> On Thu, Nov 21, 2019 at 7:01 PM Andy Lutomirski <luto@...nel.org> wrote:
>>
>> On Wed, Nov 20, 2019 at 11:52 AM syzbot
>> <syzbot+6b074f741adbd93d2df5@...kaller.appspotmail.com> wrote:
>> >
>> > syzbot has bisected this bug to:
>> >
>> > commit 0161028b7c8aebef64194d3d73e43bc3b53b5c66
>> > Author: Andy Lutomirski <luto@...nel.org>
>> > Date:   Mon May 9 22:48:51 2016 +0000
>> >
>> >      perf/core: Change the default paranoia level to 2
>> >
>> > bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=15910e86e00000
>> > start commit:   18d0eae3 Merge tag 'char-misc-4.20-rc1' of git://git.kerne..
>> > git tree:       upstream
>> > final crash:    https://syzkaller.appspot.com/x/report.txt?x=17910e86e00000
>> > console output: https://syzkaller.appspot.com/x/log.txt?x=13910e86e00000
>> > kernel config:  https://syzkaller.appspot.com/x/.config?x=342f43de913c81b9
>> > dashboard link: https://syzkaller.appspot.com/bug?extid=6b074f741adbd93d2df5
>> > syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=12482713400000
>> > C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=158fd4a3400000
>> >
>> > Reported-by: syzbot+6b074f741adbd93d2df5@...kaller.appspotmail.com
>> > Fixes: 0161028b7c8a ("perf/core: Change the default paranoia level to 2")
>> >
>> > For information about bisection process see: https://goo.gl/tpsmEJ#bisection
>>
>> Hi syzbot-
>>
>> I'm not quite sure how to tell you this in syzbotese, but I'm pretty
>> sure you've bisected this wrong.  The blamed patch makes no sense.
>
>
> Hi Andy,
>
> Three is no way to tell syzbot about this, it does not have any way to
> use this information.
> You can tell this to other recipients, though, and for the record on
> the bug report email thread. For this you can use any free form.
>
> But what makes you think this is wrong?
> From everything I see this looks like amazingly precise bisection.
> The reproducer contains perf_event_open which seems to cause the hang
> (there is a number of reports where perf_event_open hangs kernel dead
> IIRC) _and_ it contains setresuid. Which makes good match for
> "perf/core: Change the default paranoia level to 2" (for unpriv
> users).
> The bisection log also looks perfectly correct to me: no unrelated
> kernel bugs were hit along the way; the crash was always reproduced
> 100% reliably in all 10 runs; nothing else suspicious.
> I can totally imagine that your patch unmasked some latent bug, but
> it's not 100% obvious to me and in either case syzbot did the job as
> well as a robot could possibly do.

All Andy's patch did was change the default value of
sysctl_perf_event_paranoid.  Which a quick skim of the code can only
cause perf_event_open to fail.

So if perf is running as non-root aka unprivileged it might have
been affected.

That said the most likely effect that would cause a hang is for perf to
not be started and therefore it's NMI's did not happen and so something
else was free to hang.

The other possibility is something in perf_event_open goes haywire
when it attempts to start and gets permission denied.  That seems
unlikely.  Assuming that was the case Andy's change did not
touch any of the perf_event_open code.  So at most it is highlighting
a path that was broken in earlier kernels and Andy's change to
the default caused the syzbot code to take a path that was broken
much earlier.


The common sense operation to perform at this point is to realize
that the setting of sysctl_perf_event_open matters to the test and
to modify the test to set sysctl_perf_event_open before it does
more things, and then syzbot or it's keepers can track down a likely
cause for the hang.


Certainly pointing at Andy's patch gives no one any real information of
why the kernel was hanging.  It is literally changing an default value
of 1 to a default value of 2.

Eric