| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <75a3bf996be7445216df83bd4f9cd55c2cffc16a.camel@intel.com>
Date: Thu, 24 Feb 2022 01:34:37 +0000
From: "Lu, Aaron" <aaron.lu@...el.com>
To: "mgorman@...hsingularity.net" <mgorman@...hsingularity.net>
CC: "vbabka@...e.cz" <vbabka@...e.cz>,
"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
"linux-mm@...ck.org" <linux-mm@...ck.org>,
"Brouer, Jesper" <brouer@...hat.com>,
"mhocko@...nel.org" <mhocko@...nel.org>,
"akpm@...ux-foundation.org" <akpm@...ux-foundation.org>,
"dave.hansen@...ux.intel.com" <dave.hansen@...ux.intel.com>
Subject: Re: [PATCH 5/6] mm/page_alloc: Free pages in a single pass during
bulk free
On Wed, 2022-02-23 at 13:05 +0000, Mel Gorman wrote:
> On Wed, Feb 23, 2022 at 07:30:52PM +0800, Aaron Lu wrote:
> > On Mon, Feb 21, 2022 at 09:38:22PM +0800, Aaron Lu wrote:
> > > On Fri, Feb 18, 2022 at 12:20:03PM +0800, Aaron Lu wrote:
> > > > On Thu, Feb 17, 2022 at 09:31:13AM +0000, Mel Gorman wrote:
> > > > > On Thu, Feb 17, 2022 at 09:53:08AM +0800, Aaron Lu wrote:
> > > > > > > 2-socket CascadeLake (40 cores, 80 CPUs HT enabled)
> > > > > > > 5.17.0-rc3 5.17.0-rc3
> > > > > > > vanilla mm-highpcpopt-v2
> > > > > > > Hmean page_fault1-processes-2 2694662.26 ( 0.00%) 2695780.35 ( 0.04%)
> > > > > > > Hmean page_fault1-processes-5 6425819.34 ( 0.00%) 6435544.57 * 0.15%*
> > > > > > > Hmean page_fault1-processes-8 9642169.10 ( 0.00%) 9658962.39 ( 0.17%)
> > > > > > > Hmean page_fault1-processes-12 12167502.10 ( 0.00%) 12190163.79 ( 0.19%)
> > > > > > > Hmean page_fault1-processes-21 15636859.03 ( 0.00%) 15612447.26 ( -0.16%)
> > > > > > > Hmean page_fault1-processes-30 25157348.61 ( 0.00%) 25169456.65 ( 0.05%)
> > > > > > > Hmean page_fault1-processes-48 27694013.85 ( 0.00%) 27671111.46 ( -0.08%)
> > > > > > > Hmean page_fault1-processes-79 25928742.64 ( 0.00%) 25934202.02 ( 0.02%) <--
> > > > > > > Hmean page_fault1-processes-110 25730869.75 ( 0.00%) 25671880.65 * -0.23%*
> > > > > > > Hmean page_fault1-processes-141 25626992.42 ( 0.00%) 25629551.61 ( 0.01%)
> > > > > > > Hmean page_fault1-processes-172 25611651.35 ( 0.00%) 25614927.99 ( 0.01%)
> > > > > > > Hmean page_fault1-processes-203 25577298.75 ( 0.00%) 25583445.59 ( 0.02%)
> > > > > > > Hmean page_fault1-processes-234 25580686.07 ( 0.00%) 25608240.71 ( 0.11%)
> > > > > > > Hmean page_fault1-processes-265 25570215.47 ( 0.00%) 25568647.58 ( -0.01%)
> > > > > > > Hmean page_fault1-processes-296 25549488.62 ( 0.00%) 25543935.00 ( -0.02%)
> > > > > > > Hmean page_fault1-processes-320 25555149.05 ( 0.00%) 25575696.74 ( 0.08%)
> > > > > > >
> > > > > > > The differences are mostly within the noise and the difference close to
> > > > > > > $nr_cpus is negligible.
> > > > > >
> > > > > > I have queued will-it-scale/page_fault1/processes/$nr_cpu on 2 4-sockets
> > > > > > servers: CascadeLake and CooperLaker and will let you know the result
> > > > > > once it's out.
> > > > > >
> > > > >
> > > > > Thanks, 4 sockets and a later generation would be nice to cover.
> > > > >
> > > > > > I'm using 'https://github.com/hnaz/linux-mm master' and doing the
> > > > > > comparison with commit c000d687ce22("mm/page_alloc: simplify how many
> > > > > > pages are selected per pcp list during bulk free") and commit 8391e0a7e172
> > > > > > ("mm/page_alloc: free pages in a single pass during bulk free") there.
> > > > > >
> > > > >
> > > > > The baseline looks fine. It's different to what I used but the page_alloc
> > > > > shouldn't have much impact.
> > > > >
> > > > > When looking at will-it-scale, please pay attention to lower CPU counts
> > > > > as well and take account changes in standard deviation. Looking at the
> > > >
> > > > I'll also test nr_task=4/16/64 on the 4sockets CooperLake(nr_cpu=144) then.
> > > >
> > >
> > > For the record, these tests don't show any visible performance changes
> > > on CooperLake.
> >
> > One thing I just noticed is that, zone lock contention increased to some
> > extent. I'm not sure if this is worrisome so I suppose I should at least
> > mention it here.
> >
> > The nr_task=100% test on the 4 sockets Cooper Lake showed that zone lock
> > contention increased from 13.56% to 20.16% and for nr_task=16, it
> > increased from 4.75% to 6.18%.
> >
> > The reason is probably due to more code are now inside the lock and when
> > there is contention, it will make things worse. I'm aware of that
> > nr_task=100% is a rare case and this patchset is meant to improve things
> > when there is very little contention, which should be the common case.
> > So I guess that's just the tradeoff we have to make...
> >
>
> I think it's a reasonable tradeoff. The page_fault1 will-it-scale is
> an extreme case that exercises severe contention for the zone lock with
> both allocators and freeing contending for the lock at the same time. I
> think it's reasonable to optimise for the common case of completing the
> bulk freeing as quickly as possible. If anything, I think will-it-scale
> would benefit more if zone->lock was split to cover regions within a zone
> instead of protecting an entire zone which could be hundreds of GB in size.
>
Thanks for the explanation and suggestion on how to deal with zone lock
contention.
Powered by blists - more mailing lists