netdev - Re: slow performance on disk/network i/o full speed after drop

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20110901125720.GC24629@csn.ul.ie>
Date:	Thu, 1 Sep 2011 13:57:20 +0100
From:	Mel Gorman <mel@....ul.ie>
To:	Wu Fengguang <fengguang.wu@...el.com>
Cc:	Stefan Priebe - Profihost AG <s.priebe@...fihost.ag>,
	Zhu Yanhai <zhu.yanhai@...il.com>,
	Pekka Enberg <penberg@...nel.org>,
	LKML <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jens Axboe <jaxboe@...ionio.com>,
	Linux Netdev List <netdev@...r.kernel.org>,
	KOSAKI Motohiro <kosaki.motohiro@...fujitsu.com>
Subject: Re: slow performance on disk/network i/o full speed after drop_caches

On Thu, Sep 01, 2011 at 12:14:58PM +0800, Wu Fengguang wrote:
> Hi Stefan,
> 
> On Wed, Aug 31, 2011 at 03:11:02PM +0800, Stefan Priebe - Profihost AG wrote:
> > Hi Fengguang,
> > Hi Yanhai,
> > 
> > > you're abssolutely corect zone_reclaim_mode is on - but why?
> > > There must be some linux software which switches it on.
> > >
> > > ~# grep 'zone_reclaim_mode' /etc/sysctl.* -r -i
> > > ~#
> > >
> > > also
> > > ~# grep 'zone_reclaim_mode' /etc/sysctl.* -r -i
> > > ~#
> > >
> > > tells us nothing.
> > >
> > > I've then read this:
> > >
> > > "zone_reclaim_mode is set during bootup to 1 if it is determined that
> > > pages from remote zones will cause a measurable performance reduction.
> > > The page allocator will then reclaim easily reusable pages (those page
> > > cache pages that are currently not used) before allocating off node pages."
> > >
> > > Why does the kernel do that here in our case on these machines.
> > 
> > Can nobody help why the kernel in this case set it to 1?
> 
> It's determined by RECLAIM_DISTANCE.
> 
> build_zonelists():
> 
>                 /*
>                  * If another node is sufficiently far away then it is better
>                  * to reclaim pages in a zone before going off node.
>                  */
>                 if (distance > RECLAIM_DISTANCE)
>                         zone_reclaim_mode = 1;
> 
> Since Linux v3.0 RECLAIM_DISTANCE is increased from 20 to 30 by this commit.
> It may well help your case, too.
> 

Even with that, it's known that zone_reclaim() can be a disaster when
it runs into problems. This should be fixed in 3.1 by the following
commits;

[cd38b115 mm: page allocator: initialise ZLC for first zone eligible for zone_reclaim]
[76d3fbf8 mm: page allocator: reconsider zones for allocation after direct reclaim]

The description in cd38b115 has the interesting details.

-- 
Mel Gorman
SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html