linux-kernel - Re: That greedy Linux VM cache

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Thu, 30 Jan 2014 09:06:54 -0800 (PST)
From:	David Lang <david@...g.hm>
To:	Igor Podlesny <for.poige+linux@...il.com>
cc:	linux-kernel@...r.kernel.org
Subject: Re: That greedy Linux VM cache

On Fri, 31 Jan 2014, Igor Podlesny wrote:

>   Hello!
>
>   Probably every Linux newcomer's going to have concerns regarding
> low free memory and hear an explanation from Linux old fellows that's
> actually there's plenty of -- it's just cached, but when it's needed
> for applications it's gonna be used -- on demand. I also thought so
> until recently I noticed that even when free memory's is almost
> exhausted (~ 75 Mib)

that's actually quite a bit of free memory, it's very common for servers to be 
far lower than that.

> , and processes are in sleep_on_page_killable, the
> cache is somewhat like ~ 500 MiB and it's not going to return back
> what it's gained. Naturally, vm.drop_caches 3 doesn't squeeze it as
> well.

this is telling you that this data isn't clean cache that can just be dropped, 
it's dirty cache that is waiting to be written, or is otherwise locked.

rather than looking at the memory numbers, instead look at the swap numbers. If 
you are doing any noticable amount of swapping (si so in vmstat), then you are 
out of memory and the cache that can be dropped has been dropped.

this does mean that you ahve a hard time telling when you are getting close to 
running out of memory, but it's easy to see when you have.

> That drama has been happening on rather
> outdated-but-yet-still-has-2GiB-of-RAM notebook with kernel from 3.10
> till 3.12.9 (3.13 is the first release for a long time which simply
> freezes the notebook so cold, that SysRq_B's not working, but that's
> another story). Everything RAM demanding just yet crawls, load average
> is getting higher and there's no paging out, but on going disk mostly
> _read_ and a bit write activity. If vm.swaPPineSS not 0, it's swapping
> out, but not much, right now I ran Chromium (in addition to long-run
> Firefox) and only 32 MiB went to swap, load avg. ~ 7

that much read activity probably means that you are swapping pages in to use 
them, then dropping them to swap in another page, which you then drop to go back 
and fetch the first page.

David Lang

>   Again: 25 % is told (by top, free and finally /proc/meminfo) to be
> cached, but kinda greedy.
>
>   I came across similar issue report:
> http://www.spinics.net/lists/linux-btrfs/msg11723.html but still
> questions remain:
>
>   * How to analyze it? slabtop doesn't mention even 100 MiB of slab
>   * Why that's possible?
>   * The system is on Btrfs but /home is on XFS, so disk I/O might be
> related to text segment paging? But anyway this leads us to question,
> hey, there's 500 MiB free^Wcached.
>
>   While I'm thinking about moving system back to XFS...
>
>   P. S. While writing these, swapped ~ 100 MiB, and cache reduced(!)
> to 377 MiB, Firefox is mostly in "D" -- sleep_on_page_killable, so is
> Chrome, load avg. ~ 7. I had to close Skype to be able to finish that
> letter, and cached mem. now is 439 MiB. :) I know it's time to
> upgrade, but hey, cached memory is free memory, right?
>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/