linux-kernel - Re: [PATCH 00/45] hugetlb pagewalk unification

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <0f01c613-9e4f-47b6-af2b-09aa90437d90@redhat.com>
Date: Thu, 11 Jul 2024 02:15:38 +0200
From: David Hildenbrand <david@...hat.com>
To: Oscar Salvador <osalvador@...e.de>
Cc: Peter Xu <peterx@...hat.com>, Andrew Morton <akpm@...ux-foundation.org>,
 linux-kernel@...r.kernel.org, linux-mm@...ck.org,
 Muchun Song <muchun.song@...ux.dev>, SeongJae Park <sj@...nel.org>,
 Miaohe Lin <linmiaohe@...wei.com>, Michal Hocko <mhocko@...e.com>,
 Matthew Wilcox <willy@...radead.org>,
 Christophe Leroy <christophe.leroy@...roup.eu>,
 Jason Gunthorpe <jgg@...dia.com>, Ryan Roberts <ryan.roberts@....com>
Subject: Re: [PATCH 00/45] hugetlb pagewalk unification

>> (as a side note, cont-pte/cont-pmd should primarily be a hint from arch code
>> on how many entries we can batch, like we do in folio_pte_batch(); point is
>> that we want to batch also on architectures where we don't have such bits,
>> and prepare for architectures that implement various sizes of batching;
>> IMHO, having cont-pte/cont-pmd checks in common code is likely the wrong
>> approach. Again, folio_pte_batch() is where we tackled the problem
>> differently from the THP perspective)
> 
> I must say I did not check folio_pte_batch() and I am totally ignorant
> of what/how it does things.
> I will have a look.
> 
>> I have an idea for a better page table walker API that would try batching
>> most entries (under one PTL), and walkers can just register for the types
>> they want. Hoping I will find some time to at least scetch the user
>> interface soon.
>>
>> That doesn't mean that this should block your work, but the
>> cont-pte/cont/pmd hugetlb stuff is really nasty to handle here, and I don't
>> particularly like where this is going.
> 
> Ok, let me take a step back then.
> Previous versions of that RFC did not handle cont-{pte-pmd} wide in the
> open, so let me go back to the drawing board and come up with something
> that does not fiddle with cont- stuff in that way.
> 
> I might post here a small diff just to see if we are on the same page.
> 
> As usual, thanks a lot for your comments David!

Feel free to reach out to discuss ways forward. I think we should

(a) move to the automatic cont-pte setting as done for THPs via
     set_ptes().
(b) Batching PTE updates at all relevant places, so we get no change in
     behavior: cont-pte bit will remain set.
(c) Likely remove the use of cont-pte bits in hugetlb code for anything
     that is not a present folio (i.e., where automatic cont-pte bit
     setting would never set it). Migration entries might require
     thought (we can easily batch to achieve the same thing, but the
     behavior of hugetlb likely differs to the generic way of handling
     migration entries on multiple ptes: reference the folio vs.
     the respective subpages of the folio).

Then we are essentially replacing what the current hugetlb_ functions do 
by the common way it is being done for THP (which does not exist for 
cont-pmd yet ... ). The only real alternative is special casing hugetlb 
all over the place to still call the hugetlb_* functions.

:( it would all be easier without that hugetlb cont-pte/cont-pmd usage.

CCing Ryan so he's aware.

-- 
Cheers,

David / dhildenb