linux-kernel - Re: [PATCHv5 06/12] x86/boot/compressed: Handle unaccepted memory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Thu, 2 Jun 2022 14:51:37 +0200
From:   "Gupta, Pankaj" <pankaj.gupta@....com>
To:     Dionna Amalie Glaze <dionnaglaze@...gle.com>
Cc:     "Xu, Min M" <min.m.xu@...el.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>,
        Borislav Petkov <bp@...e.de>,
        "Gao, Jiaqi" <jiaqi.gao@...el.com>,
        Michael Roth <michael.roth@....com>,
        Borislav Petkov <bp@...en8.de>,
        "Kirill A. Shutemov" <kirill@...temov.name>,
        "Lutomirski, Andy" <luto@...nel.org>,
        "Christopherson,, Sean" <seanjc@...gle.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        "Rodel, Jorg" <jroedel@...e.de>, Ard Biesheuvel <ardb@...nel.org>,
        Andi Kleen <ak@...ux.intel.com>,
        Kuppuswamy Sathyanarayanan 
        <sathyanarayanan.kuppuswamy@...ux.intel.com>,
        David Rientjes <rientjes@...gle.com>,
        Vlastimil Babka <vbabka@...e.cz>,
        Tom Lendacky <thomas.lendacky@....com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Peter Zijlstra <peterz@...radead.org>,
        Paolo Bonzini <pbonzini@...hat.com>,
        Ingo Molnar <mingo@...hat.com>,
        Varad Gautam <varad.gautam@...e.com>,
        Dario Faggioli <dfaggioli@...e.com>,
        "Hansen, Dave" <dave.hansen@...el.com>,
        Mike Rapoport <rppt@...nel.org>,
        David Hildenbrand <david@...hat.com>,
        "x86@...nel.org" <x86@...nel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        "linux-coco@...ts.linux.dev" <linux-coco@...ts.linux.dev>,
        "linux-efi@...r.kernel.org" <linux-efi@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCHv5 06/12] x86/boot/compressed: Handle unaccepted memory


> The memory accounting in Linux is probably the issue. Both times I ran
> the test were from a freshly booted VM. The test parses the output of
> $(free -k) to determine the amount of free memory it should allocate
> and write/read from, with a given stride of pages to skip before
> touching the next page.
> 
> We grab the third column of numbers from the Mem output that looks like this
> 
>                 total        used        free      shared  buff/cache   available
> Mem:        65856604     4128688    48558952       11208    13168964    60942928
> Swap:        1953788      118124     1835664
> 
> So my workstation has 48558952 free bytes. We take that, give it to
> memtouch to allocate that much anonymous memory rounded down to the
> nearest MB with mmap and randomly read/write the buffer.
> 
> For an 8GB machine, the UEFI will have the initial 0-0xA000 memory and
> 0x10_0000 to 0xC00_0000 (beginning of mmio hole) prevalidated. The
> next 5GB is classified as the UEFI v2.9 memory type
> EFI_RESOURCE_MEMORY_UNACCEPTED, 0x1_4000_000 to 0x2_0000_0000.
> The Linux e820 map should see that range as unaccepted rather than
> EFI_CONVENTIONAL_MEMORY (i.e., EDK2's EFI_RESOURCE_SYSTEM_MEMORY), but
> I think it needs to be accounted as free conventional memory.

AFAIU the unaccepted memory also stays in buddy (first via slow path) 
and should be accounted automatically in free?

> 
> So when I see 2044MB free vs 7089MB free in my VMs, the two are
> roughly 5GB different.

Is it possible all memory got allocated with memblock? Maybe some 
variable tests to validate with '/proc/meminfo | grep UnacceptedMem' 
would give you more clue.

Thanks,
Pankaj