linux-kernel - Re: memory offline infinite loop after soft offline

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191017093410.GA19973@hori.linux.bs1.fc.nec.co.jp>
Date:   Thu, 17 Oct 2019 09:34:10 +0000
From:   Naoya Horiguchi <n-horiguchi@...jp.nec.com>
To:     Michal Hocko <mhocko@...nel.org>
CC:     Qian Cai <cai@....pw>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-mm@...ck.org" <linux-mm@...ck.org>,
        David Hildenbrand <david@...hat.com>,
        Mike Kravetz <mike.kravetz@...cle.com>
Subject: Re: memory offline infinite loop after soft offline

On Mon, Oct 14, 2019 at 10:39:14AM +0200, Michal Hocko wrote:
> On Fri 11-10-19 17:32:44, Qian Cai wrote:
> > # /opt/ltp/runtest/bin/move_pages12
> > move_pages12.c:263: INFO: Free RAM 258988928 kB
> > move_pages12.c:281: INFO: Increasing 2048kB hugepages pool on node 0 to 4
> > move_pages12.c:291: INFO: Increasing 2048kB hugepages pool on node 8 to 4
> > move_pages12.c:207: INFO: Allocating and freeing 4 hugepages on node 0
> > move_pages12.c:207: INFO: Allocating and freeing 4 hugepages on node 8
> > move_pages12.c:197: PASS: Bug not reproduced
> > move_pages12.c:197: PASS: Bug not reproduced
> > 
> > for mem in $(ls -d /sys/devices/system/memory/memory*); do
> >         echo offline > $mem/state
> >         echo online > $mem/state
> > done
> > 
> > That LTP move_pages12 test will first madvise(MADV_SOFT_OFFLINE) for a range.
> > Then, one of "echo offline" will trigger an infinite loop in __offline_pages()
> > here,
> > 
> > 		/* check again */
> > 		ret = walk_system_ram_range(start_pfn, end_pfn - start_pfn,
> > 					    NULL, check_pages_isolated_cb);
> > 	} while (ret);
> > 
> > because check_pages_isolated_cb() always return -EBUSY from
> > test_pages_isolated(),
> > 
> > 
> > 	pfn = __test_page_isolated_in_pageblock(start_pfn, end_pfn,
> > 						skip_hwpoisoned_pages);
> >         ...
> >         return pfn < end_pfn ? -EBUSY : 0;
> > 
> > The root cause is in __test_page_isolated_in_pageblock() where "pfn" is always
> > less than "end_pfn" because the associated page is not a PageBuddy.
> > 
> > 	while (pfn < end_pfn) {
> > 	...
> > 		else
> > 			break;
> > 
> > 	return pfn;
> 
> Hmm, this is interesting. I would expect that this would hit the
> previous branch
> 	if (skip_hwpoisoned_pages && PageHWPoison(page))
> and skip over hwpoisoned page. But I cannot seem to find that we would
> mark all tail pages HWPoison as well and so any tail page seem to
> confuse __test_page_isolated_in_pageblock.
> 
> Oscar is rewriting the hwpoison implementation but I am not sure
> how/whether he is handling this case differently. Naoya, shouldn't we do
> the following at least?

My appology for late response.

> ---
> diff --git a/mm/page_isolation.c b/mm/page_isolation.c
> index 89c19c0feadb..5fb3fee16fde 100644
> --- a/mm/page_isolation.c
> +++ b/mm/page_isolation.c
> @@ -274,7 +274,7 @@ __test_page_isolated_in_pageblock(unsigned long pfn, unsigned long end_pfn,
>  			 * simple way to verify that as VM_BUG_ON(), though.
>  			 */
>  			pfn += 1 << page_order(page);
> -		else if (skip_hwpoisoned_pages && PageHWPoison(page))
> +		else if (skip_hwpoisoned_pages && PageHWPoison(compound_head(page)))
>  			/* A HWPoisoned page cannot be also PageBuddy */
>  			pfn++;
>  		else

This fix looks good to me. The original code only addresses hwpoisoned 4kB-page,
we seem to have this issue since the following commit,

  commit b023f46813cde6e3b8a8c24f432ff9c1fd8e9a64
  Author: Wen Congyang <wency@...fujitsu.com>
  Date:   Tue Dec 11 16:00:45 2012 -0800
  
      memory-hotplug: skip HWPoisoned page when offlining pages

and extension of LTP coverage finally discovered this.

Thanks,
Naoya Horiguchi