linux-kernel - Re: [PATCH 0/8][for -mm] mem

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ab3f9b940804011635g2de833d0l44558f78a1cce1e5@mail.gmail.com>
Date:	Tue, 1 Apr 2008 16:35:06 -0700
From:	"Tom May" <tom@...may.com>
To:	linux-mm@...ck.org, linux-kernel@...r.kernel.org
Cc:	"KOSAKI Motohiro" <kosaki.motohiro@...fujitsu.com>,
	linux-fsdevel@...r.kernel.org
Subject: Re: [PATCH 0/8][for -mm] mem_notify v6

On Sat, Feb 9, 2008 at 8:19 AM, KOSAKI Motohiro
<kosaki.motohiro@...fujitsu.com> wrote:
> Hi
>
>  The /dev/mem_notify is low memory notification device.
>  it can avoid swappness and oom by cooperationg with the user process.
>
>  the Linux Today article is very nice description. (great works by Jake Edge)
>  http://www.linuxworld.com/news/2008/020508-kernel.html
>
>  <quoted>
>  When memory gets tight, it is quite possible that applications have memory
>  allocated—often caches for better performance—that they could free.
>  After all, it is generally better to lose some performance than to face the
>  consequences of being chosen by the OOM killer.
>  But, currently, there is no way for a process to know that the kernel is
>  feeling memory pressure.
>  The patch provides a way for interested programs to monitor the /dev/mem_notify
>   file to be notified if memory starts to run low.
>  </quoted>
>
>
>  You need not be annoyed by OOM any longer :)
>  please any comments!

Thanks for this patch set!  I ported it to 2.6.23.9 and tried it, on a
system with no swap since I'm evaluating this for an embedded system.
In practice, the criterion it uses for notifications wasn't sufficient to avoid
memory problems, including OOM, in a cyclic allocate/notify/free
sequence which is probably typical.

I tried it with a real-world program that, among other things, mmaps
anonymous pages and touches them at a reasonable speed until it gets
notified via /dev/mem_notify, releases most of them with
madvise(MADV_DONTNEED), then loops to start the cycle again.

What tends to happen is that I do indeed get notifications via
/dev/mem_notify when the kernel would like to be swapping, at which
point I free memory.  But the notifications come at a time when the
kernel needs memory, and it gets the memory by discarding some Cached
or Mapped memory (I can see these decreasing in /proc/meminfo with
each notification).  With each mmap/notify/madvise cycle the Cached
and Mapped memory gets smaller, until eventually while I'm touching
pages the kernel can't find enough memory and will either invoke the
OOM killer or return ENOMEM from syscalls.  This is precisely the
situation I'm trying to avoid by using /dev/mem_notify.

The criterion of "notify when the kernel would like to swap" feels
correct, but in addition I seem to need something like "notify when
cached+mapped+free memory is getting low".

I'll need to be looking into doing this, so any comments or ideas are
welcome.

Thanks,
.tom
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/