| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20090305004850.GA6045@localhost>
Date: Thu, 5 Mar 2009 08:48:50 +0800
From: Wu Fengguang <fengguang.wu@...el.com>
To: Markus <M4rkusXXL@....de>
Cc: "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
Zdenek Kabelac <zkabelac@...hat.com>,
"linux-mm@...ck.org" <linux-mm@...ck.org>,
Lukas Hejtmanek <xhejtman@....muni.cz>
Subject: Re: drop_caches ...
On Wed, Mar 04, 2009 at 08:47:41PM +0200, Markus wrote:
> Am Mittwoch, 4. März 2009 schrieb Zdenek Kabelac:
> > Markus napsal(a):
> > >>>>>>> The memory mapped pages won't be dropped in this way.
> > >>>>>>> "cat /proc/meminfo" will show you the number of mapped pages.
> > >>>>>> # sync ; echo 3 > /proc/sys/vm/drop_caches ; free -m ;
> > >>>> cat /proc/meminfo
> > >>>>>> total used free shared buffers
> > >>>>>> cached
> > >>>>>> Mem: 3950 3262 688 0 0
> > >>>>>> 359
> > >>>>>> -/+ buffers/cache: 2902 1047
> > >>>>>> Swap: 5890 1509 4381
> > >>>>>> MemTotal: 4045500 kB
> > >>>>>> MemFree: 705180 kB
> > >>>>>> Buffers: 508 kB
> > >>>>>> Cached: 367748 kB
> > >>>>>> SwapCached: 880744 kB
> > >>>>>> Active: 1555032 kB
> > >>>>>> Inactive: 1634868 kB
> > >>>>>> Active(anon): 1527100 kB
> > >>>>>> Inactive(anon): 1607328 kB
> > >>>>>> Active(file): 27932 kB
> > >>>>>> Inactive(file): 27540 kB
> > >>>>>> Unevictable: 816 kB
> > >>>>>> Mlocked: 0 kB
> > >>>>>> SwapTotal: 6032344 kB
> > >>>>>> SwapFree: 4486496 kB
> > >>>>>> Dirty: 0 kB
> > >>>>>> Writeback: 0 kB
> > >>>>>> AnonPages: 2378112 kB
> > >>>>>> Mapped: 52196 kB
> > >>>>>> Slab: 65640 kB
> > >>>>>> SReclaimable: 46192 kB
> > >>>>>> SUnreclaim: 19448 kB
> > >>>>>> PageTables: 28200 kB
> > >>>>>> NFS_Unstable: 0 kB
> > >>>>>> Bounce: 0 kB
> > >>>>>> WritebackTmp: 0 kB
> > >>>>>> CommitLimit: 8055092 kB
> > >>>>>> Committed_AS: 4915636 kB
> > >>>>>> VmallocTotal: 34359738367 kB
> > >>>>>> VmallocUsed: 44580 kB
> > >>>>>> VmallocChunk: 34359677239 kB
> > >>>>>> DirectMap4k: 3182528 kB
> > >>>>>> DirectMap2M: 1011712 kB
> > >>>>>>
> > >>>>>> The cached reduced to 359 MB (after the dropping).
> > >>>>>> I dont know where to read the "number of mapped pages".
> > >>>>>> "Mapped" is about 51 MB.
> > >>>>> Does your tmpfs store lots of files?
> > >>>> Dont think so:
> > >>>>
> > >>>> # df -h
> > >>>> Filesystem Size Used Avail Use% Mounted on
> > >>>> /dev/md6 14G 8.2G 5.6G 60% /
> > >>>> udev 10M 304K 9.8M 3% /dev
> > >>>> cachedir 4.0M 100K 4.0M 3% /lib64/splash/cache
> > >>>> /dev/md4 19G 15G 3.1G 83% /home
> > >>>> /dev/md3 8.3G 4.5G 3.9G 55% /usr/portage
> > >>>> shm 2.0G 0 2.0G 0% /dev/shm
> > >>>> /dev/md1 99M 19M 76M 20% /boot
> > >>>>
> > >>>> I dont know what exactly all that memory is used for. It varies
> > >>>> from about 300 MB to up to one GB.
> > >>>> Tell me where to look and I will!
> > >>> So you don't have lots of mapped pages(Mapped=51M) or tmpfs files.
> > > It's strange to me that there are so many undroppable cached pages(Cached=359M),
> > > and most of them lie out of the LRU queue(Active+Inactive file=53M)...
> > >>> Anyone have better clues on these 'hidden' pages?
> > >> Maybe try this:
> > >>
> > >> cat /proc/`pidof X`/smaps | grep drm | wc -l
> > >>
> > >> you will see some growing numbers.
> > >>
> > >> Also check cat /proc/dri/0/gem_objects
> > >> there should be some number # object bytes - which should be close
> to
> > > your
> > >> missing cached pages.
> > >>
> > >>
> > >> If you are using Intel GEM driver - there is some unlimited caching
> > > issue
> > >> see: http://bugs.freedesktop.org/show_bug.cgi?id=20404
> > >>
> > > # cat /proc/`pidof X`/smaps | grep drm | wc -l
> > > 0
> > > # cat /proc/dri/0/gem_objects
> > > cat: /proc/dri/0/gem_objects: No such file or directory
> > >
> > > I use Xorg 1.3 with an nvidia gpu. Dont know if I use a "Intel GEM
> > > driver".
> > >
> >
> >
> > Are you using binary driver from NVidia ??
> > Maybe you should ask authors of this binary blob ?
> >
> > Could you try to use for a while Vesa driver to see, if you are able
> to get
> > same strange results ?
>
> I rebooted in console without the nvidia-module loaded and have the same
> results (updated to 2.6.28.7 btw):
> # sync ; echo 3 > /proc/sys/vm/drop_caches ; free -m ; cat /proc/meminfo
> total used free shared buffers
> cached
> Mem: 3950 1647 2303 0 0
> 924
> -/+ buffers/cache: 722 3228
> Swap: 5890 0 5890
> MemTotal: 4045444 kB
> MemFree: 2358944 kB
> Buffers: 544 kB
> Cached: 946624 kB
> SwapCached: 0 kB
> Active: 1614756 kB
> Inactive: 7632 kB
> Active(anon): 1602476 kB
> Inactive(anon): 0 kB
> Active(file): 12280 kB
> Inactive(file): 7632 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> SwapTotal: 6032344 kB
> SwapFree: 6032344 kB
> Dirty: 72 kB
> Writeback: 32 kB
> AnonPages: 675224 kB
> Mapped: 17756 kB
> Slab: 19936 kB
> SReclaimable: 9652 kB
> SUnreclaim: 10284 kB
> PageTables: 8296 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 8055064 kB
> Committed_AS: 3648088 kB
> VmallocTotal: 34359738367 kB
> VmallocUsed: 10616 kB
> VmallocChunk: 34359716459 kB
> DirectMap4k: 6080 kB
> DirectMap2M: 4188160 kB
Markus, you may want to try this patch, it will have better chance to figure out
the hidden file pages.
1) apply the patch and recompile kernel with CONFIG_PROC_FILECACHE=m
2) after booting:
modprobe filecache
cp /proc/filecache filecache-`date +'%F'`
3) send us the copied file, it will list all cached files, including
the normally hidden ones.
Thanks,
Fengguang
View attachment "filecache-2.6.28.patch" of type "text/x-diff" (31993 bytes)
Powered by blists - more mailing lists