linux-kernel - Re: [PATCH v9 0/8] stg mail -e --version=v9 \

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <6b4451e9-7f62-bb84-1604-a4aebc907766@redhat.com>
Date:   Wed, 11 Sep 2019 18:02:46 +0200
From:   David Hildenbrand <david@...hat.com>
To:     Michal Hocko <mhocko@...nel.org>
Cc:     "Michael S. Tsirkin" <mst@...hat.com>,
        Alexander Duyck <alexander.h.duyck@...ux.intel.com>,
        Alexander Duyck <alexander.duyck@...il.com>,
        virtio-dev@...ts.oasis-open.org, kvm list <kvm@...r.kernel.org>,
        Catalin Marinas <catalin.marinas@....com>,
        Dave Hansen <dave.hansen@...el.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Matthew Wilcox <willy@...radead.org>,
        linux-mm <linux-mm@...ck.org>,
        Andrew Morton <akpm@...ux-foundation.org>, will@...nel.org,
        linux-arm-kernel@...ts.infradead.org,
        Oscar Salvador <osalvador@...e.de>,
        Yang Zhang <yang.zhang.wz@...il.com>,
        Pankaj Gupta <pagupta@...hat.com>,
        Konrad Rzeszutek Wilk <konrad.wilk@...cle.com>,
        Nitesh Narayan Lal <nitesh@...hat.com>,
        Rik van Riel <riel@...riel.com>, lcapitulino@...hat.com,
        "Wang, Wei W" <wei.w.wang@...el.com>,
        Andrea Arcangeli <aarcange@...hat.com>, ying.huang@...el.com,
        Paolo Bonzini <pbonzini@...hat.com>,
        Dan Williams <dan.j.williams@...el.com>,
        Fengguang Wu <fengguang.wu@...el.com>,
        "Kirill A. Shutemov" <kirill.shutemov@...ux.intel.com>
Subject: Re: [PATCH v9 0/8] stg mail -e --version=v9 \

>> Something slightly similar is being performed by Nitesh's patch set. On
>> every free of a certain granularity, he records it in the bitmap. These
>> bits are "hints of free pages".
>>
>> A thread then walks over the bitmap and tries to allocate the "hints".
>> If the pages were already reused, the bit is silently cleared.
>>
>> Instead of allocating/freeing, we could only try to isolate the
>> pageblock, then test if free. (One of the usual issues to work around is
>> MAX_ORDER-1 crossing pageblocks, that might need special care)
> 
> OK, cool that I have reinvented the wheel ;). Allocation is indeed not
> necessary as long as pages are isolated because nobody will allocate
> them.

It's always good if you come to the same conclusion ;)

>  
>> I think you should have a look at the rough idea of Nitesh's patch set
>> to see if something like that is going into a better direction. The
>> bitmap part is in place to do bulk reporting and avoid duplicate reports.
> 
> Let's see how much time I can find for that in my endless inbox whack a mole.

Can totally understand - it's only a single patch.

>  
>> I think main points we want (and what I am missing from callback idea
>> being discussed) are
>> 1. Do bulk reporting only when a certain threshold is reached
> 
> Is a time based approach too coarse?

Usually that's then another parameter to fine tune, something to avoid
when just reporting continuously. I wouldn't say it's a no go, but at
least I would prefer right now to do it continuously.

> 
>> 2. Report only bigger granularities (especially, avoid THP splits in the
>> hypervisor - >= 2MB proofed to be effective)
> 
> the callback has supported order based scan in some of its iteration.

Missed that. But yeah, the other points are more important :)

> 
>> 3. Avoid reporting what has just been reported.
> 
> Is the overhead of checking a pfn range in a bitmask that much of an
> overhead to really care?

It's all about remembering what has already been reported. Nitesh solved
that via the bitmap. So he does exactly that. If we can optimize the
bitmap out - perfect - but I don't see an easy way to do that :)

> 
>> 4. Continuously report, not the "one time report everything" approach.
> 
> So you mean the allocator reporting this rather than an external code to
> poll right? I do not know, how much this is nice to have than must have?

I guess it is debatable - but I don't consider this one of the
fundamental issues. How to identify what to report and remember what has
already been reported is the main issue. Polling vs. notification is
just the cherry on top - IMHO.

-- 

Thanks,

David / dhildenb