linux-kernel - Re: [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CALCETrUJ2qU9i0CQmMyW-We-RodN7pZRFEVeb5TT9Bz1jUTa_A@mail.gmail.com>
Date:   Wed, 22 Mar 2017 09:59:29 -0700
From:   Andy Lutomirski <luto@...nel.org>
To:     Thomas Garnier <thgarnie@...gle.com>
Cc:     Ingo Molnar <mingo@...nel.org>, Andy Lutomirski <luto@...nel.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        kernel test robot <xiaolong.ye@...el.com>,
        Alexander Potapenko <glider@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Andrey Ryabinin <aryabinin@...tuozzo.com>,
        Ard Biesheuvel <ard.biesheuvel@...aro.org>,
        Boris Ostrovsky <boris.ostrovsky@...cle.com>,
        Borislav Petkov <bp@...e.de>,
        Chris Wilson <chris@...is-wilson.co.uk>,
        Christian Borntraeger <borntraeger@...ibm.com>,
        Dmitry Vyukov <dvyukov@...gle.com>,
        Frederic Weisbecker <fweisbec@...il.com>,
        Jiri Kosina <jikos@...nel.org>, Joerg Roedel <joro@...tes.org>,
        Jonathan Corbet <corbet@....net>,
        Josh Poimboeuf <jpoimboe@...hat.com>,
        Juergen Gross <jgross@...e.com>,
        Kees Cook <keescook@...omium.org>,
        Len Brown <len.brown@...el.com>,
        Lorenzo Stoakes <lstoakes@...il.com>,
        "Luis R . Rodriguez" <mcgrof@...nel.org>,
        Matt Fleming <matt@...eblueprint.co.uk>,
        Michal Hocko <mhocko@...e.com>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Paul Gortmaker <paul.gortmaker@...driver.com>,
        Pavel Machek <pavel@....cz>,
        Peter Zijlstra <peterz@...radead.org>,
        Radim Krčmář <rkrcmar@...hat.com>,
        "Rafael J . Wysocki" <rjw@...ysocki.net>,
        Rusty Russell <rusty@...tcorp.com.au>,
        Stanislaw Gruszka <sgruszka@...hat.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Tim Chen <tim.c.chen@...ux.intel.com>,
        Vitaly Kuznetsov <vkuznets@...hat.com>,
        zijun_hu <zijun_hu@....com>, LKML <linux-kernel@...r.kernel.org>,
        Stephen Rothwell <sfr@...b.auug.org.au>, LKP <lkp@...org>
Subject: Re: [lkp-robot] [x86] 69218e4799: BUG:kernel_hang_in_boot_stage

On Wed, Mar 22, 2017 at 9:38 AM, Thomas Garnier <thgarnie@...gle.com> wrote:
> On Wed, Mar 22, 2017 at 9:33 AM, Andy Lutomirski <luto@...capital.net> wrote:
>> On Wed, Mar 22, 2017 at 12:36 AM, Ingo Molnar <mingo@...nel.org> wrote:
>>>
>>> * Thomas Garnier <thgarnie@...gle.com> wrote:
>>>
>>>> >  static inline void setup_fixmap_gdt(int cpu)
>>>> >  {
>>>> >         __set_fixmap(get_cpu_gdt_ro_index(cpu),
>>>> > -                    __pa(get_cpu_gdt_rw(cpu)), pg_fixmap_gdt_flags);
>>>> > +                    slow_virt_to_phys(get_cpu_gdt_rw(cpu)),
>>>> > +                    pg_fixmap_gdt_flags);
>>>> >  }
>>>> >
>>>> >  /* Load the original GDT from the per-cpu structure */
>>>> >
>>>> > This makes UP boot for me, but SMP (2 cpus) is still busted.
>>>>
>>>> This change fixed boot for me:
>>>>
>>>> diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
>>>> index b65155cc3760..4e30707d9f9a 100644
>>>> --- a/arch/x86/include/asm/fixmap.h
>>>> +++ b/arch/x86/include/asm/fixmap.h
>>>> @@ -104,7 +104,9 @@ enum fixed_addresses {
>>>>         FIX_GDT_REMAP_BEGIN,
>>>>         FIX_GDT_REMAP_END = FIX_GDT_REMAP_BEGIN + NR_CPUS - 1,
>>>>
>>>> -       __end_of_permanent_fixed_addresses,
>>>> +       __end_of_permanent_fixed_addresses =
>>>> +               (FIX_GDT_REMAP_END + PTRS_PER_PTE - 1) &
>>>> +               -PTRS_PER_PTE,
>>>>
>>>> Just ensure PKMAP_BASE & FIX_WP_TEST are on a different PMD.
>>>>
>>>> I don't think that the right fix but it might help understand the
>>>> exact root cause.
>>>
>>> Could this be related to the permission bits in the PMD itself getting out of sync
>>> with the PTEs? WP test marks a page writable/unwritable, and maybe we mess up the
>>> restoration. If they are on separate PMDs then this is worked around because the
>>> fixmap GDT is on a separate PMD.
>>>
>>
>> I don't think so.  I think it's a pair of bugs related to the way that
>> percpu areas are virtually mapped.
>>
>> Bug 1: __pa is totally bogus on percpu pointers.  Oddly, we have one
>> older instance of exactly the same bug (on the same GDT address) in
>> the kernel.  I'll send a patch.
>>
>> Bug 2: Nothing syncs a freshly-set-up CPU's percpu area into
>> initial_page_table.  This makes access to the gdt fail in
>> startup_32_smp.  This looks like a longstanding bug, and I don't see
>> what it has to do with Thomas' series.  I'm still mulling over what to
>> do about it.
>
> Why do you think padding the fixmap also fix the problem? That's the
> thing I don't get.
>
> With the padding the PA is now correct and the memcmp check also
> succeed. That's odd.
>

Not sure.  There are some complicated heuristics in the percpu code
that determine how it's allocated, and the padding might be affecting
those heuristics.

--Andy