| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 22 Jul 2015 18:33:26 +0530
From: PINTU KUMAR <pintu.k@...sung.com>
To: 'Mel Gorman' <mgorman@...e.de>
Cc: akpm@...ux-foundation.org, corbet@....net, vbabka@...e.cz,
gorcunov@...nvz.org, mhocko@...e.cz, emunson@...mai.com,
kirill.shutemov@...ux.intel.com, standby24x7@...il.com,
hannes@...xchg.org, vdavydov@...allels.com, hughd@...gle.com,
minchan@...nel.org, tj@...nel.org, rientjes@...gle.com,
xypron.glpk@....de, dzickus@...hat.com, prarit@...hat.com,
ebiederm@...ssion.com, rostedt@...dmis.org, uobergfe@...hat.com,
paulmck@...ux.vnet.ibm.com, iamjoonsoo.kim@....com,
ddstreet@...e.org, sasha.levin@...cle.com, koct9i@...il.com,
cj@...ux.com, opensource.ganesh@...il.com, vinmenon@...eaurora.org,
linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
linux-mm@...ck.org, linux-pm@...r.kernel.org, qiuxishi@...wei.com,
Valdis.Kletnieks@...edu, cpgs@...sung.com, pintu_agarwal@...oo.com,
vishnu.ps@...sung.com, rohit.kr@...sung.com, iqbal.ams@...sung.com,
pintu.ping@...il.com, pintu.k@...look.com
Subject: RE: [PATCH v3 1/1] kernel/sysctl.c: Add /proc/sys/vm/shrink_memory
feature
Dear Mel, thank you very much for your comments and suggestions.
I will drop this one and look on further improving direct_reclaim and
compaction.
Just few more comments below before I close.
Also, during this patch, I feel that the hibernation_mode part in
shrink_all_memory can be corrected.
So, can I separately submit the below patch?
That is instead of hard-coding the hibernation_mode, we can get hibernation
status using:
system_entering_hibernation()
Please let me know your suggestion about this changes.
-#ifdef CONFIG_HIBERNATION
+#if defined CONFIG_HIBERNATION || CONFIG_SHRINK_MEMORY
/*
* Try to free `nr_to_reclaim' of memory, system-wide, and return the number of
* freed pages.
@@ -3576,12 +3580,16 @@ unsigned long shrink_all_memory(unsigned long
nr_to_reclaim)
.may_writepage = 1,
.may_unmap = 1,
.may_swap = 1,
- .hibernation_mode = 1,
};
struct zonelist *zonelist = node_zonelist(numa_node_id(), sc.gfp_mask);
struct task_struct *p = current;
unsigned long nr_reclaimed;
+ if (system_entering_hibernation())
+ sc.hibernation_mode = 1;
+ else
+ sc.hibernation_mode = 0;
+
p->flags |= PF_MEMALLOC;
lockdep_set_current_reclaim_state(sc.gfp_mask);
reclaim_state.reclaimed_slab = 0;
@@ -3597,6 +3605,28 @@ unsigned long shrink_all_memory(unsigned long
nr_to_reclaim)
}
#endif /* CONFIG_HIBERNATION */
> -----Original Message-----
> From: Mel Gorman [mailto:mgorman@...e.de]
> Sent: Monday, July 20, 2015 11:26 PM
> To: PINTU KUMAR
> Cc: akpm@...ux-foundation.org; corbet@....net; vbabka@...e.cz;
> gorcunov@...nvz.org; mhocko@...e.cz; emunson@...mai.com;
> kirill.shutemov@...ux.intel.com; standby24x7@...il.com;
> hannes@...xchg.org; vdavydov@...allels.com; hughd@...gle.com;
> minchan@...nel.org; tj@...nel.org; rientjes@...gle.com;
> xypron.glpk@....de; dzickus@...hat.com; prarit@...hat.com;
> ebiederm@...ssion.com; rostedt@...dmis.org; uobergfe@...hat.com;
> paulmck@...ux.vnet.ibm.com; iamjoonsoo.kim@....com; ddstreet@...e.org;
> sasha.levin@...cle.com; koct9i@...il.com; cj@...ux.com;
> opensource.ganesh@...il.com; vinmenon@...eaurora.org; linux-
> doc@...r.kernel.org; linux-kernel@...r.kernel.org; linux-mm@...ck.org; linux-
> pm@...r.kernel.org; qiuxishi@...wei.com; Valdis.Kletnieks@...edu;
> cpgs@...sung.com; pintu_agarwal@...oo.com; vishnu.ps@...sung.com;
> rohit.kr@...sung.com; iqbal.ams@...sung.com; pintu.ping@...il.com;
> pintu.k@...look.com
> Subject: Re: [PATCH v3 1/1] kernel/sysctl.c: Add /proc/sys/vm/shrink_memory
> feature
>
> On Mon, Jul 20, 2015 at 09:43:02PM +0530, PINTU KUMAR wrote:
> > Hi,
> >
> > Thank you all for reviewing the patch and providing your valuable
> > comments and suggestions.
> > During the ELC conference many people suggested to release the patch
> > to mainline, so this patch, to get others opinion.
> >
>
> Unfortunately, in my opinion it runs the risk of creating a different set of
> problems. Either it needs to be run frequently to keep memory free which
incurs
> one set of penalties or it is used too late when there are
> unmovable/unreclaimable pages preventing allocations succeeding in which case
> you are back at the original problem.
Yes, I completely agree with you that it needs to be invoked at the right time.
Running it too late is of no benefit.
> I see what you did and why it would work in some cases
> but I think the main reason it works is because it's run frequently
> enough so memory is never used.
Yes, we ran frequently, but not so frequently and only when required.
Actually, it gives us best result when calling shrink_memory plus compaction
together,
once after boot, and once during order-4 failure from kernel, or during suspend
state.
It reduced the slowpath count drastically (during 30 application launch test).
VMSTAT WITHOUT WITH
slowpath_entered 16659 1859
allocstall 298 149
pageoutrun 2699 1108
compact_stall 244 37
nr_free_cma 2560 2505
Anyways, I agree that if reclaimable pages or SWAP free is not enough, it does
not
yield good results.
> Grouping pages by mobility actually took
> advantage of a similar property when it increased min_free_kbytes but that was
> much more limited than adding a giant hammer for userspace to reclaim the
> world.
>
> > If you have any more suggestions to experiment and verify please let me
know.
> >
>
> I believe I already did. If it's high-order reliability that is important then
you need
> to either reserve the memory or look at protecting the pages using grouping
> pages by mobility. I pointed out what series to look at and the leader
explains
> how it could be adjusted further for the embedded case if necessary.
Thanks. I would definitely look into grouping pages by mobility and those
series.
>
> If it's latency you are interested in then reclaim/compaction needs to be
modified
> to be more aggressive when it is somehow detected that the high-order
> allocation must succeed for functional correctness. In that case the
relational
> starting point would be to look at should_continue_reclaim and how it relates
to
> compaction.
>
Thanks. Definitely I will do a deep dive into should_continue_reclaim.
> > The suggestion was only to open up the shrink_all_memory API for some use
> cases.
> >
> > I am not saying that it needs to be called continuously. It can be
> > used only on certain condition and only when deemed necessary.
> > The same technique is already used in hibernation to reduce the RAM
> > snapshot image size.
>
> Reducing memory usage is not the same as guaranteeing that high-order pages
> are available for allocation.
>
> --
> Mel Gorman
> SUSE Labs
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists