[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20210113052209.75531-5-songmuchun@bytedance.com>
Date: Wed, 13 Jan 2021 13:22:07 +0800
From: Muchun Song <songmuchun@...edance.com>
To: mike.kravetz@...cle.com, akpm@...ux-foundation.org
Cc: n-horiguchi@...jp.nec.com, ak@...ux.intel.com, mhocko@...e.cz,
linux-mm@...ck.org, linux-kernel@...r.kernel.org,
Muchun Song <songmuchun@...edance.com>
Subject: [PATCH v4 4/6] mm: hugetlb: retry dissolve page when hitting race
There is a race between dissolve_free_huge_page() and put_page().
Theoretically, we should return -EBUSY when we encounter this race.
In fact, we have a chance to successfully dissolve the page if we
do a retry. Because the race window is quite small. If we seize
this opportunity, it is an optimization for increasing the success
rate of dissolving page.
Signed-off-by: Muchun Song <songmuchun@...edance.com>
---
mm/hugetlb.c | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 4a9011e12175..898e4ea43e13 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -1772,6 +1772,7 @@ int dissolve_free_huge_page(struct page *page)
{
int rc = -EBUSY;
+retry:
/* Not to disrupt normal path by vainly holding hugetlb_lock */
if (!PageHuge(page))
return 0;
@@ -1793,8 +1794,23 @@ int dissolve_free_huge_page(struct page *page)
* We should make sure that the page is already on the free list
* when it is dissolved.
*/
- if (unlikely(!PageHugeFreed(head)))
- goto out;
+ if (unlikely(!PageHugeFreed(head))) {
+ spin_unlock(&hugetlb_lock);
+
+ /*
+ * Theoretically, we should return -EBUSY when we
+ * encounter this race. In fact, we have a chance
+ * to successfully dissolve the page if we do a
+ * retry. Because the race window is quite small.
+ * If we seize this opportunity, it is an optimization
+ * for increasing the success rate of dissolving page.
+ */
+ while (PageHeadHuge(head) && !PageHugeFreed(head)) {
+ cond_resched();
+ cpu_relax();
+ }
+ goto retry;
+ }
/*
* Move PageHWPoison flag from head page to the raw error page,
--
2.11.0
Powered by blists - more mailing lists