linux-kernel - Re: Hardcore trashing without any swap

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20100614111246.3149aba0.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Mon, 14 Jun 2010 11:12:46 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	László Monda <laci@...da.hu>
Cc:	Ed Tomlinson <edt@....ca>, Alan Cox <alan@...rguk.ukuu.org.uk>,
	linux-kernel <linux-kernel@...r.kernel.org>
Subject: Re: Hardcore trashing without any swap

On Fri, 11 Jun 2010 23:38:58 +0200
László Monda <laci@...da.hu> wrote:

> On Fri, Jun 11, 2010 at 3:47 PM, Ed Tomlinson <edt@....ca> wrote:
> > On Friday 11 June 2010 08:53:50 László Monda wrote:
> >> On Fri, Jun 11, 2010 at 2:16 PM, Alan Cox <alan@...rguk.ukuu.org.uk> wrote:
> >> > On Fri, 11 Jun 2010 02:10:33 +0200
> >> > László Monda <laci@...da.hu> wrote:
> >> >
> >> >> Hi List,
> >> >>
> >> >> The problem I'm facing with is very simple, yet extremely irritating
> >> >> in nature.  I have a laptop with 4G RAM and I don't use any swap.
> >> >> Whenever the RAM is full my system keeps trashing.  This makes X and
> >> >> SSH completely unresponsive for about a hour then a bunch of processes
> >> >> gets killed and it's usable again.
> >> >>
> >> >> How is possible that my system is trashing even though I don't use any swap?
> >> >
> >> > Because you don't have any swap. Its having to dump stuff it doesn't want
> >> > to like bits of applications that it can retrieve back from disk.
> >>
> >> I can read what you wrote but cannot really understand it.  Please
> >> tell me where my logic fails:
> >>
> >> No swap -> no dedicated space on disk to dump stuff -> no disk I/O
> >> should happen at all
> >
> > No.  This is not the case.  If the vm needs memory it will discard pages from that are
> > backed by objects _not_ stored in swap - like executables.  Only if there is nothing to
> > discard will it start killing...  That being said you need to read up on what Alan's
> > suggestion below does - or add a swapfile (which works nearly as well as a swap partition
> > now).
> 
> By reading your reply I think I understood what's going on.
> 
> Previously I thought that trashing can only happen because the kernel
> discards pages to swap which I thought is the only reason why disk I/O
> can happen in such a case.  It didn't made sense to me because I don't
> use swap.
> 
> The other possibility that occured to me is the following scenario:
> 1) Process A grows really big and fills up the RAM.
> 2) The kernel discards code pages from process B in favor of process A.
> 3) Process A has some RAM and keeps growing further until it almost
> fills up the RAM.
> 4) Process B gets scheduled and needs to be paged in which makes
> process A paged out, hence disk I/O occurs.
> 5) Process A gets scheduled and needs to be paged in which makes
> process B paged out, hence disk I/O occurs.
> 6) Go to 4)
> 
> This is not truly an infinite loop because the memory gradually gets
> filled up in 4) and 5) but it happens really slowly because process
> code has to be paged in upon every rescheduling.
> 
> So I've just realized that trashing cannot only happen due to paging
> out to swap.  It can also happen due to simply discarding code pages
> and paging them in later.
> 
> Is the above scenario valid in my case?
> 
> I suppose that the OOM killer could kill process A in step 3) right
> away but the scheduler is fast and the 4)-5) loop keeps making disk
> I/O for a very long time until process A exhausts the memory and the
> OOM killer intervenes eventually.
> 
> I've also realized that it's probably impossible to truly reliably
> foresee trashing so that's why there are no out of the box solutions
> for my problem.  The best hack I can think of is a daemon that
> monitors memory usage in about every 10 milisec and kills the biggest
> process if something seems to go wrong before trashing can begin.
> 
If you know possible memory eaters, can't be memory cgroup a help ? 
You can keep some important tasks by isolating workloads.

Thanks,
-Kame

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/