linux-kernel - Re: KASAN vs. boot-time switching between 4- and 5-level paging

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170710141713.7aox3edx6o7lrrie@node.shutemov.name>
Date:   Mon, 10 Jul 2017 17:17:13 +0300
From:   "Kirill A. Shutemov" <kirill@...temov.name>
To:     Dmitry Vyukov <dvyukov@...gle.com>
Cc:     Andrey Ryabinin <aryabinin@...tuozzo.com>,
        Alexander Potapenko <glider@...gle.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "x86@...nel.org" <x86@...nel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>,
        "H. Peter Anvin" <hpa@...or.com>, Andi Kleen <ak@...ux.intel.com>,
        Dave Hansen <dave.hansen@...el.com>,
        Andy Lutomirski <luto@...capital.net>,
        linux-arch@...r.kernel.org,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        LKML <linux-kernel@...r.kernel.org>,
        kasan-dev <kasan-dev@...glegroups.com>
Subject: Re: KASAN vs. boot-time switching between 4- and 5-level paging

On Mon, Jul 10, 2017 at 02:43:17PM +0200, Dmitry Vyukov wrote:
> On Mon, Jul 10, 2017 at 2:33 PM, Kirill A. Shutemov
> <kirill@...temov.name> wrote:
> > On Thu, Jun 01, 2017 at 05:56:30PM +0300, Andrey Ryabinin wrote:
> >> On 05/29/2017 03:46 PM, Andrey Ryabinin wrote:
> >> > On 05/29/2017 02:45 PM, Andrey Ryabinin wrote:
> >> >>>>>> Looks like KASAN will be a problem for boot-time paging mode switching.
> >> >>>>>> It wants to know CONFIG_KASAN_SHADOW_OFFSET at compile-time to pass to
> >> >>>>>> gcc -fasan-shadow-offset=. But this value varies between paging modes...
> >> >>>>>>
> >> >>>>>> I don't see how to solve it. Folks, any ideas?
> >> >>>>>
> >> >>>>> +kasan-dev
> >> >>>>>
> >> >>>>> I wonder if we can use the same offset for both modes. If we use
> >> >>>>> 0xFFDFFC0000000000 as start of shadow for 5 levels, then the same
> >> >>>>> offset that we use for 4 levels (0xdffffc0000000000) will also work
> >> >>>>> for 5 levels. Namely, ending of 5 level shadow will overlap with 4
> >> >>>>> level mapping (both end at 0xfffffbffffffffff), but 5 level mapping
> >> >>>>> extends towards lower addresses. The current 5 level start of shadow
> >> >>>>> is actually close -- 0xffd8000000000000 and it seems that the required
> >> >>>>> space after it is unused at the moment (at least looking at mm.txt).
> >> >>>>> So just try to move it to 0xFFDFFC0000000000?
> >> >>>>>
> >> >>>>
> >> >>>> Yeah, this should work, but note that 0xFFDFFC0000000000 is not PGDIR aligned address. Our init code
> >> >>>> assumes that kasan shadow stars and ends on the PGDIR aligned address.
> >> >>>> Fortunately this is fixable, we'd need two more pages for page tables to map unaligned start/end
> >> >>>> of the shadow.
> >> >>>
> >> >>> I think we can extend the shadow backwards (to the current address),
> >> >>> provided that it does not affect shadow offset that we pass to
> >> >>> compiler.
> >> >>
> >> >> I thought about this. We can round down shadow start to 0xffdf000000000000, but we can't
> >> >> round up shadow end, because in that case shadow would end at 0xffffffffffffffff.
> >> >> So we still need at least one more page to cover unaligned end.
> >> >
> >> > Actually, I'm wrong here. I assumed that we would need an additional page to store p4d entries,
> >> > but in fact we don't need it, as such page should already exist. It's the same last pgd where kernel image
> >> > is mapped.
> >> >
> >>
> >>
> >> Something like bellow might work. It's just a proposal to demonstrate the idea, so some code might look ugly.
> >> And it's only build-tested.
> >
> > [Sorry for loong delay.]
> >
> > The patch works for me for legacy boot. But it breaks EFI boot with
> > 5-level paging. And I struggle to understand why.
> >
> > What I see is many page faults at mm/kasan/kasan.c:758 --
> > "DEFINE_ASAN_LOAD_STORE(4)". Handling one of them I get double-fault at
> > arch/x86/kernel/head_64.S:298 -- "pushq %r14", which ends up with triple
> > fault.
> >
> > Any ideas?
> 
> 
> Just playing the role of the rubber duck:
>  - what is the fault address?
>  - is it within the shadow range?
>  - was the shadow mapped already?

I misread trace. The initial fault is at arch/x86/kernel/head_64.S:270,
which is ".endr" in definition of early_idt_handler_array.

The fault address for all three faults is 0xffffffff7ffffff8, which is
outside shadow range. It's just before kernel text mapping.

Codewise, it happens in load_ucode_bsp() -- after kasan_early_init(), but
before kasan_init().

-- 
 Kirill A. Shutemov