linux-kernel - Re: [PATCH mm-unstable v1] mm/hugetlb_vmemmap: fix memory loads ordering

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAOUHufa4PmMY8Yg2=2p+-HWLO6mjqMou0aROf0-jhA3BfC5_1Q@mail.gmail.com>
Date: Wed, 8 Jan 2025 00:32:48 -0700
From: Yu Zhao <yuzhao@...gle.com>
To: Muchun Song <muchun.song@...ux.dev>
Cc: Andrew Morton <akpm@...ux-foundation.org>, David Hildenbrand <david@...hat.com>, 
	Mateusz Guzik <mjguzik@...il.com>, "Matthew Wilcox (Oracle)" <willy@...radead.org>, linux-mm@...ck.org, 
	linux-kernel@...r.kernel.org, Will Deacon <will@...nel.org>
Subject: Re: [PATCH mm-unstable v1] mm/hugetlb_vmemmap: fix memory loads ordering

On Tue, Jan 7, 2025 at 1:41 AM Muchun Song <muchun.song@...ux.dev> wrote:
>
>
>
> > On Jan 7, 2025, at 12:35, Yu Zhao <yuzhao@...gle.com> wrote:
> >
> > Using x86_64 as an example, for a 32KB struct page[] area describing a
> > 2MB hugeTLB, HVO reduces the area to 4KB by the following steps:
> > 1. Split the (r/w vmemmap) PMD mapping the area into 512 (r/w) PTEs;
> > 2. For the 8 PTEs mapping the area, remap PTE 1-7 to the page mapped
> >   by PTE 0, and at the same time change the permission from r/w to
> >   r/o;
> > 3. Free the pages PTE 1-7 used to map, hence the reduction from 32KB
> >   to 4KB.
> >
> > However, the following race can happen due to improperly memory loads
> > ordering:
> >  CPU 1 (HVO)                     CPU 2 (speculative PFN walker)
> >
> >  page_ref_freeze()
> >  synchronize_rcu()
> >                                  rcu_read_lock()
> >                                  page_is_fake_head() is false
> >  vmemmap_remap_pte()
> >  XXX: struct page[] becomes r/o
> >
> >  page_ref_unfreeze()
> >                                  page_ref_count() is not zero
> >
> >                                  atomic_add_unless(&page->_refcount)
> >                                  XXX: try to modify r/o struct page[]
> >
> > Specifically, page_is_fake_head() must be ordered after
> > page_ref_count() on CPU 2 so that it can only return true for this
> > case, to avoid the later attempt to modify r/o struct page[].
> >
> > This patch adds the missing memory barrier and makes the tests on
> > page_is_fake_head() and page_ref_count() done in the proper order.
> >
> > Fixes: bd225530a4c7 ("mm/hugetlb_vmemmap: fix race with speculative PFN walkers")
> > Reported-by: Will Deacon <will@...nel.org>
> > Closes: https://lore.kernel.org/20241128142028.GA3506@willie-the-truck/
> > Signed-off-by: Yu Zhao <yuzhao@...gle.com>
> > ---
> > include/linux/page-flags.h | 2 +-
> > include/linux/page_ref.h   | 8 ++++++--
> > 2 files changed, 7 insertions(+), 3 deletions(-)
> >
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h
> > index 691506bdf2c5..6b8ecf86f1b6 100644
> > --- a/include/linux/page-flags.h
> > +++ b/include/linux/page-flags.h
> > @@ -212,7 +212,7 @@ static __always_inline const struct page *page_fixed_fake_head(const struct page
> > * cold cacheline in some cases.
> > */
> >       if (IS_ALIGNED((unsigned long)page, PAGE_SIZE) &&
> > -         test_bit(PG_head, &page->flags)) {
> > +         test_bit_acquire(PG_head, &page->flags)) {
> >               /*
> >                * We can safely access the field of the @page[1] with PG_head
> >                * because the @page is a compound page composed with at least
> > diff --git a/include/linux/page_ref.h b/include/linux/page_ref.h
> > index 8c236c651d1d..5becea98bd79 100644
> > --- a/include/linux/page_ref.h
> > +++ b/include/linux/page_ref.h
> > @@ -233,8 +233,12 @@ static inline bool page_ref_add_unless(struct page *page, int nr, int u)
> >       bool ret = false;
> >
> >       rcu_read_lock();
> > -     /* avoid writing to the vmemmap area being remapped */
> > -     if (!page_is_fake_head(page) && page_ref_count(page) != u)
> > +     /*
> > +      * To avoid writing to the vmemmap area remapped into r/o in parallel,
> > +      * the page_ref_count() test must precede the page_is_fake_head() test
> > +      * so that test_bit_acquire() in the latter is ordered after the former.
> > +      */
> > +     if (page_ref_count(page) != u && !page_is_fake_head(page))
>
> IIUC, we need to insert a memory barrier between page_ref_count() and page_is_fake_head().
> Specifically, accessing between page->_refcount and page->flags. So we should insert a
> read memory barrier here, right?

Correct, i.e., page_ref_count(page) != u; smp_rmb(); !page_is_fake_head(page).

> But I saw you added an acquire barrier in page_fixed_fake_head(),
> I don't understand why an acquire barrier could stop the CPU reordering the accessing
> between them. What am I missing here?

A load-acquire on page->_refcount would be equivalent to the smp_rmb()
above. But apparently I used on page->flags because I misremembered
whether a load-acquire inserts the equivalent smp_rmb() before or
after (it's after, not before). Will fix this in v2.