linux-kernel - Re: [RFC] mm:change meminfo cached calculation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LSU.2.11.1501061747540.2041@eggly.anvils>
Date:	Tue, 6 Jan 2015 18:03:47 -0800 (PST)
From:	Hugh Dickins <hughd@...gle.com>
To:	Andrew Morton <akpm@...ux-foundation.org>
cc:	Jerome Marchand <jmarchan@...hat.com>,
	Hugh Dickins <hughd@...gle.com>,
	"Wang, Yalin" <Yalin.Wang@...ymobile.com>,
	"minchan@...nel.org" <minchan@...nel.org>,
	Konstantin Khlebnikov <koct9i@...il.com>,
	"Kirill A. Shutemov" <kirill@...temov.name>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-mm@...ck.org" <linux-mm@...ck.org>,
	"linux-arm-kernel@...ts.infradead.org" 
	<linux-arm-kernel@...ts.infradead.org>,
	"n-horiguchi@...jp.nec.com" <n-horiguchi@...jp.nec.com>,
	"pintu.k@...sung.com" <pintu.k@...sung.com>
Subject: Re: [RFC] mm:change meminfo cached calculation

On Tue, 6 Jan 2015, Andrew Morton wrote:
> On Tue, 6 Jan 2015 17:04:33 -0800 (PST) Hugh Dickins <hughd@...gle.com> wrote:
> > On Tue, 6 Jan 2015, Andrew Morton wrote:
> > > On Fri, 26 Dec 2014 19:56:49 +0800 "Wang, Yalin" <Yalin.Wang@...ymobile.com> wrote:
> > > 
> > > > This patch subtract sharedram from cached,
> > > > sharedram can only be swap into swap partitions,
> > > > they should be treated as swap pages, not as cached pages.
> > > > 
> > > > ...
> > > >
> > > > --- a/fs/proc/meminfo.c
> > > > +++ b/fs/proc/meminfo.c
> > > > @@ -45,7 +45,7 @@ static int meminfo_proc_show(struct seq_file *m, void *v)
> > > >  	committed = percpu_counter_read_positive(&vm_committed_as);
> > > >  
> > > >  	cached = global_page_state(NR_FILE_PAGES) -
> > > > -			total_swapcache_pages() - i.bufferram;
> > > > +			total_swapcache_pages() - i.bufferram - i.sharedram;
> > > >  	if (cached < 0)
> > > >  		cached = 0;
> > > 
> > > Documentation/filesystems/proc.txt says
> > > 
> > > :      Cached: in-memory cache for files read from the disk (the
> > > :              pagecache).  Doesn't include SwapCached
> > > 
> > > So yes, I guess it should not include shmem.
> > > 
> > > And why not do this as well?
> > > 
> > > 
> > > --- a/Documentation/filesystems/proc.txt~mm-change-meminfo-cached-calculation-fix
> > > +++ a/Documentation/filesystems/proc.txt
> > > @@ -811,7 +811,7 @@ MemAvailable: An estimate of how much me
> > >       Buffers: Relatively temporary storage for raw disk blocks
> > >                shouldn't get tremendously large (20MB or so)
> > >        Cached: in-memory cache for files read from the disk (the
> > > -              pagecache).  Doesn't include SwapCached
> > > +              pagecache).  Doesn't include SwapCached or Shmem.
> > >    SwapCached: Memory that once was swapped out, is swapped back in but
> > >                still also is in the swapfile (if memory is needed it
> > >                doesn't need to be swapped out AGAIN because it is already
> > 
> > Whoa.  Changes of this kind would have made good sense about 14 years ago.
> > And there's plenty more which would benefit from having anon/shmem/file
> > properly distinguished.  But how can we make such a change now,
> > breaking everything that has made its own sense of these counts?
> 
> That's what I was wondering, but I was having some trouble picking a
> situation where it mattered much.

If it doesn't matter, then we don't need to change it.

> What's the problematic scenario
> here?  Userspace that is taking Cached, saying "that was silly" and
> subtracting Shmem from it by hand?

Someone a long time ago saw "that was silly", worked out it was because
of Shmem, adjusted their scripts or whatever accordingly, and has run
happily ever since.

> 
> I suppose that as nobody knows we should err on the side of caution and
> leave this alone.  But the situation is pretty sad - it would be nice
> to make the code agree with the documentation at least.

By all means fix the documentation.  And work on a /proc/meminfo.2015
which has sensibly differentiated counts (and probably omits that
wonderful Linux 2.2-compatible "Buffers").

But there's more to do than I can think of.  Cc'ing Jerome who has a
particular interest in this (no, I haven't forgotten his patches,
but nor have I had a moment to reconsider them).

Hugh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/