linux-kernel - Re: KASAN vs. boot-time switching between 4- and 5-level paging

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <03A6D7ED-300C-4431-9EB5-67C7A3EA4A2E@amacapital.net>
Date:   Mon, 10 Jul 2017 08:56:37 -0700
From:   Andy Lutomirski <luto@...capital.net>
To:     "Kirill A. Shutemov" <kirill@...temov.name>
Cc:     Dmitry Vyukov <dvyukov@...gle.com>,
        Andrey Ryabinin <aryabinin@...tuozzo.com>,
        Alexander Potapenko <glider@...gle.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "x86@...nel.org" <x86@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>, Andi Kleen <ak@...ux.intel.com>,
        Dave Hansen <dave.hansen@...el.com>,
        linux-arch@...r.kernel.org,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        kasan-dev <kasan-dev@...glegroups.com>
Subject: Re: KASAN vs. boot-time switching between 4- and 5-level paging



> On Jul 10, 2017, at 7:17 AM, Kirill A. Shutemov <kirill@...temov.name> wrote:
> 
>> On Mon, Jul 10, 2017 at 02:43:17PM +0200, Dmitry Vyukov wrote:
>> On Mon, Jul 10, 2017 at 2:33 PM, Kirill A. Shutemov
>> <kirill@...temov.name> wrote:
>>> On Thu, Jun 01, 2017 at 05:56:30PM +0300, Andrey Ryabinin wrote:
>>>>> On 05/29/2017 03:46 PM, Andrey Ryabinin wrote:
>>>>> On 05/29/2017 02:45 PM, Andrey Ryabinin wrote:
>>>>>>>>>> Looks like KASAN will be a problem for boot-time paging mode switching.
>>>>>>>>>> It wants to know CONFIG_KASAN_SHADOW_OFFSET at compile-time to pass to
>>>>>>>>>> gcc -fasan-shadow-offset=. But this value varies between paging modes...
>>>>>>>>>> 
>>>>>>>>>> I don't see how to solve it. Folks, any ideas?
>>>>>>>>> 
>>>>>>>>> +kasan-dev
>>>>>>>>> 
>>>>>>>>> I wonder if we can use the same offset for both modes. If we use
>>>>>>>>> 0xFFDFFC0000000000 as start of shadow for 5 levels, then the same
>>>>>>>>> offset that we use for 4 levels (0xdffffc0000000000) will also work
>>>>>>>>> for 5 levels. Namely, ending of 5 level shadow will overlap with 4
>>>>>>>>> level mapping (both end at 0xfffffbffffffffff), but 5 level mapping
>>>>>>>>> extends towards lower addresses. The current 5 level start of shadow
>>>>>>>>> is actually close -- 0xffd8000000000000 and it seems that the required
>>>>>>>>> space after it is unused at the moment (at least looking at mm.txt).
>>>>>>>>> So just try to move it to 0xFFDFFC0000000000?
>>>>>>>>> 
>>>>>>>> 
>>>>>>>> Yeah, this should work, but note that 0xFFDFFC0000000000 is not PGDIR aligned address. Our init code
>>>>>>>> assumes that kasan shadow stars and ends on the PGDIR aligned address.
>>>>>>>> Fortunately this is fixable, we'd need two more pages for page tables to map unaligned start/end
>>>>>>>> of the shadow.
>>>>>>> 
>>>>>>> I think we can extend the shadow backwards (to the current address),
>>>>>>> provided that it does not affect shadow offset that we pass to
>>>>>>> compiler.
>>>>>> 
>>>>>> I thought about this. We can round down shadow start to 0xffdf000000000000, but we can't
>>>>>> round up shadow end, because in that case shadow would end at 0xffffffffffffffff.
>>>>>> So we still need at least one more page to cover unaligned end.
>>>>> 
>>>>> Actually, I'm wrong here. I assumed that we would need an additional page to store p4d entries,
>>>>> but in fact we don't need it, as such page should already exist. It's the same last pgd where kernel image
>>>>> is mapped.
>>>>> 
>>>> 
>>>> 
>>>> Something like bellow might work. It's just a proposal to demonstrate the idea, so some code might look ugly.
>>>> And it's only build-tested.
>>> 
>>> [Sorry for loong delay.]
>>> 
>>> The patch works for me for legacy boot. But it breaks EFI boot with
>>> 5-level paging. And I struggle to understand why.
>>> 
>>> What I see is many page faults at mm/kasan/kasan.c:758 --
>>> "DEFINE_ASAN_LOAD_STORE(4)". Handling one of them I get double-fault at
>>> arch/x86/kernel/head_64.S:298 -- "pushq %r14", which ends up with triple
>>> fault.
>>> 
>>> Any ideas?
>> 
>> 
>> Just playing the role of the rubber duck:
>> - what is the fault address?
>> - is it within the shadow range?
>> - was the shadow mapped already?
> 
> I misread trace. The initial fault is at arch/x86/kernel/head_64.S:270,
> which is ".endr" in definition of early_idt_handler_array.
> 
> The fault address for all three faults is 0xffffffff7ffffff8, which is
> outside shadow range. It's just before kernel text mapping.
> 
> Codewise, it happens in load_ucode_bsp() -- after kasan_early_init(), but
> before kasan_init().

My theory is that, in 5 level mode, the early IDT code isn't all mapped in the page tables.  This could sometimes be papered over by lazy page table setup, but lazy setup can't handle faults in the page fault code or data structures.

EFI sometimes uses separate page tables, which could contribute.

> 
> -- 
> Kirill A. Shutemov