linux-kernel - RE: [linux-mm] Drastic increase in application memory usage with Kernel version upgrade

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 10 Aug 2016 18:56:02 +0530
From:	PINTU KUMAR <pintu.k@...sung.com>
To:	'Pavel Machek' <pavel@....cz>, koct9i@...il.com
Cc:	'Minchan Kim' <minchan@...nel.org>, linux-kernel@...r.kernel.org,
	linux-mm@...ck.org, jaejoon.seo@...sung.com, jy0.jeon@...sung.com,
	vishnu.ps@...sung.com, chulspro.kim@...sung.com
Subject: RE: [linux-mm] Drastic increase in application memory usage with
 Kernel version upgrade

Hi,

> -----Original Message-----
> From: Pavel Machek [mailto:pavel@....cz]
> Sent: Saturday, August 06, 2016 2:20 AM
> To: PINTU KUMAR
> Cc: 'Minchan Kim'; linux-kernel@...r.kernel.org; linux-mm@...ck.org;
> jaejoon.seo@...sung.com; jy0.jeon@...sung.com; vishnu.ps@...sung.com
> Subject: Re: [linux-mm] Drastic increase in application memory usage with
Kernel
> version upgrade
> 
> On Fri 2016-08-05 20:17:36, PINTU KUMAR wrote:
> > Hi,
> 
> > > On Fri, Aug 05, 2016 at 10:26:37AM +0530, PINTU KUMAR wrote:
> > > > Hi All,
> > > >
> > > > For one of our ARM embedded product, we recently updated the
> > > > Kernel version from 3.4 to 3.18 and we noticed that the same 
> > > > application memory usage  (PSS value) gone up by ~10% and for 
> > > > some cases it even crossed ~50%. There is no change in platform 
> > > > part. All platform component was  built with ARM 32-bit toolchain.
> > > > However, the Kernel is changed from 32-bit to 64-bit.
> > > >
> > > > Is upgrading Kernel version and moving from 32-bit to 64-bit is
> > > > such a risk?
> > > > After the upgrade, what can we do further to reduce the
> > > > application memory usage ?
> > > > Is there any other factor that will help us to improve without
> > > > major modifications in platform ?
> > > >
> > > > As a proof, we did a small experiment on our Ubuntu-32 bit machine.
> > > > We upgraded Ubuntu Kernel version from 3.13 to 4.01 and we
> > > > observed the following:
> > > > ------------------------------------------------------------------
> > > > |UBUNTU-32 bit	|Kernel 3.13	|Kernel 4.03	|DIFF	|
> > > > |CALCULATOR PSS	|6057 KB	|6466 KB	|409 KB	|
> > > > ------------------------------------------------------------------
> > > > So, just by upgrading the Kernel version: PSS value for calculator
> > > > is increased by 409KB.
> > > >
> > > > If anybody knows any in-sight about it please point out more
> > > > details about the root cause.
> > >
> > > One of culprit is [8c6e50b0290c, mm: introduce vm_ops->map_pages()].
> > Ok. Thank you for your reply.
> > So, if I revert this patch, will the memory usage be decreased for the
> > processes with Kernel 3.18 ?
> 
> I guess you should try it...
> 
Thanks for the reply and confirmation.
Our exact kernel version is: 3.18.14
And, we already have this patch:
/*
mm: do not call do_fault_around for non-linear fault
Ingo Korb reported that "repeated mapping of the same file on tmpfs
using remap_file_pages sometimes triggers a BUG at mm/filemap.c:202 when
the process exits".
He bisected the bug to d7c1755179b8 ("mm: implement ->map_pages for
shmem/tmpfs"), although the bug was actually added by commit
8c6e50b0290c ("mm: introduce vm_ops->map_pages()").
*/

So, I guess, reverting this patch (8c6e50b0290c), is not required ?
But, still we have memory usage issue.

> You may want to try the same kernel version, once in 32-bit and once in 64-bit
> version. And you may consider moving to recent kernel.
> 
Sorry, but moving to higher kernel version is not possible as of now. 
As it involves other BSP changes.
We can only upgrades the relevant patches.
However, as I said earlier, we even found the difference on Ubuntu 32-bit 
after moving from Kernel-3.13 to 4.0.
So, I guess, this problem exists even in higher kernel version too.

> Yes, 64-bit kernel will increase memory usage _of kernel_, but...
> 
Ok, But ?
Will it be major culprit ? Or higher kernel version is major culprit ?

> 
> 	Pavel
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures)
http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html