linux-kernel - Re: [PATCH] kernel: fs: drop_caches: add dds drop_caches

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:	Tue, 16 Feb 2016 10:37:06 -0800 (PST)
From:	Nag Avadhanam <nag@...co.com>
To:	Vladimir Davydov <vdavydov@...tuozzo.com>
cc:	"Theodore Ts'o" <tytso@....edu>,
	"Daniel Walker (danielwa)" <danielwa@...co.com>,
	Dave Chinner <david@...morbit.com>,
	Alexander Viro <viro@...iv.linux.org.uk>,
	"Khalid Mughal (khalidm)" <khalidm@...co.com>,
	"xe-kernel@...ernal.cisco.com" <xe-kernel@...ernal.cisco.com>,
	"dave.hansen@...el.com" <dave.hansen@...el.com>,
	"hannes@...xchg.org" <hannes@...xchg.org>,
	"riel@...hat.com" <riel@...hat.com>,
	Jonathan Corbet <corbet@....net>,
	"linux-doc@...r.kernel.org" <linux-doc@...r.kernel.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>
Subject: Re: [PATCH] kernel: fs: drop_caches: add dds drop_caches_count

On Tue, 16 Feb 2016, Vladimir Davydov wrote:

> On Tue, Feb 16, 2016 at 02:58:04AM +0000, Nag Avadhanam (nag) wrote:
>> We have a class of platforms that are essentially swap-less embedded
>> systems that have limited memory resources (2GB and less).
>>
>> There is a need to implement early alerts (before the OOM killer kicks in)
>> based on the current memory usage so admins can take appropriate steps (do
>> not initiate provisioning operations but support existing services,
>> de-provision certain services, etc. based on the extent of memory usage in
>> the system) .
>>
>> There is also a general need to let end users know the available memory so
>> they can determine if they can enable new services (helps in planning).
>>
>> These two depend upon knowing approximate (accurate within few 10s of MB)
>> memory usage within the system. We want to alert admins before system
>> exhibits any thrashing behaviors.
>
> Have you considered using /proc/kpageflags for counting such pages? It
> should already export all information about memory pages you might need,
> e.g. which pages are mapped, which are anonymous, which are inactive,
> basically all page flags and even more. Moreover, you can even determine
> the set of pages that are really read/written by processes - see
> /sys/kernel/mm/page_idle/bitmap. On such a small machine scanning the
> whole pfn range should be pretty cheap, so you might find this API
> acceptable.

Thanks Vladimir. I came across the pagmemap interface sometime ago. I
was not sure if its mainstream. I think this should allow userspace 
VM scan (scans might take a bit longer). Will try it.

We could avoid the scans altogether.

The need plainly put is, inform the admins of these swapless embedded systems 
of the available memory.

If we can reliably and efficiently maintain counts of file pages 
(inactive and active) mapped into the address spaces of active user space 
processes, this need can be met. "Mapped" of /proc/meminfo does not seem 
to be a direct fit for this purpose (I need to understand this better). 
If I know for sure "Mapped" does not count device and the kernel pages 
mapped into the user space, then I can employ it gainfully for this need.

(Cached - Shmem - <mapped file/binary pages of active processes>) gives me
reclaimable file pages. If I can determine that then I can add that to MemFree 
and determine the available memory.

Thanks,
nag

>
> Thanks,
> Vladimir
>