linux-kernel - Re: [PATCH v10] mm: compaction: handle incorrect MIGRATE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-id: <201206111654.48701.b.zolnierkie@samsung.com>
Date:	Mon, 11 Jun 2012 16:54:48 +0200
From:	Bartlomiej Zolnierkiewicz <b.zolnierkie@...sung.com>
To:	Minchan Kim <minchan@...nel.org>
Cc:	linux-mm@...ck.org, linux-kernel@...r.kernel.org,
	Hugh Dickins <hughd@...gle.com>,
	KOSAKI Motohiro <kosaki.motohiro@...il.com>,
	Dave Jones <davej@...hat.com>, Cong Wang <amwang@...hat.com>,
	Markus Trippelsdorf <markus@...ppelsdorf.de>,
	Mel Gorman <mgorman@...e.de>, Rik van Riel <riel@...hat.com>,
	Marek Szyprowski <m.szyprowski@...sung.com>,
	Kyungmin Park <kyungmin.park@...sung.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [PATCH v10] mm: compaction: handle incorrect MIGRATE_UNMOVABLE
 type pageblocks

On Monday 11 June 2012 15:39:16 Minchan Kim wrote:
> On Mon, Jun 11, 2012 at 12:43:14PM +0200, Bartlomiej Zolnierkiewicz wrote:
> > On Monday 11 June 2012 03:26:49 Minchan Kim wrote:
> > > Hi Bartlomiej,
> > > 
> > > On 06/08/2012 05:46 PM, Bartlomiej Zolnierkiewicz wrote:
> > > 
> > > > 
> > > > Hi,
> > > > 
> > > > This version is much simpler as it just uses __count_immobile_pages()
> > > > instead of using its own open coded version and it integrates changes
> > > 
> > > 
> > > That's a good idea. I don't have noticed that function is there.
> > > When I look at the function, it has a problem, too.
> > > Please, look at this.
> > > 
> > > https://lkml.org/lkml/2012/6/10/180
> > > 
> > > If reviewer is okay that patch, I would like to resend your patch based on that. 
> > 
> > Ok, I would later merge all changes into v11 and rebase on top of your patch.
> > 
> > > > from Minchan Kim (without page_count change as it doesn't seem correct
> > > 
> > > 
> > > Why do you think so?
> > > If it isn't correct, how can you prevent racing with THP page freeing?
> > 
> > After seeing the explanation for the previous fix it is all clear now.
> > 
> > > > and __count_immobile_pages() does the check in the standard way; if it
> > > > still is a problem I think that removing 1st phase check altogether
> > > > would be better instead of adding more locking complexity).
> > > > 
> > > > The patch also adds compact_rescued_unmovable_blocks vmevent to vmstats
> > > > to make it possible to easily check if the code is working in practice.
> > > 
> > > 
> > > I think that part should be another patch.
> > > 
> > > 1. Adding new vmstat would be arguable so it might interrupt this patch merging.
> > 
> > Why would it be arguable?  It seems non-intrusive and obvious to me.
> 
> Once you add new vmstat, someone can make another dependent code in userspace.
> It means your new vmstat would become a new ABI so we should be careful.

I know about it but I doubt that it will be ever used by the user-space
for other purpose than showing kernel statistics (even that is unlikely).

> > 
> > > 2. New vmstat adding is just for this patch is effective or not in real practice
> > >    so if we prove it in future, let's revert the vmstat. Separating it would make it
> > >    easily.
> > 
> > I would like to add this vmstat permanently, not only for the testing period..
> 
> "I would like to add this vmstat permanently" isn't logical at all.
> You should mention why we need such vmstat and how administrator can parse it/
> handle it if he needs.

I quickly went through vmstat history and I see rationales like this:
"Optional patch, but useful for development and understanding the system."
for adding new vmstat counters.  The new counter falls into this category.

compact_rescued_unmovable_blocks shows the number of MIGRATE_UNMOVABLE
pageblocks converted back to MIGRATE_MOVABLE type by the memory compaction
code.  Non-zero values indicate that large kernel-originated allocations
of MIGRATE_UNMOVABLE type happen in the system and need special handling 
from the memory compaction code.

> If we have Documentation/vmstat.txt, you should have written it down. Sigh.

But we don't have vmstat.txt even though we have a lot of vmstat counters. :(

Best regards,
--
Bartlomiej Zolnierkiewicz
Samsung Poland R&D Center
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/