linux-kernel - Re: backing up ext4 fs, system unresponsive, thrashing like crazy even though swap is unused

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Mon, 3 Dec 2012 12:43:42 -0500
From:	Theodore Ts'o <tytso@....edu>
To:	Dimitrios Apostolou <jimis@....net>
Cc:	Roland Eggner <edvx1@...temanalysen.net>,
	linux-kernel@...r.kernel.org,
	Catalin Marinas <catalin.marinas@....com>
Subject: Re: backing up ext4 fs, system unresponsive, thrashing like crazy
 even though swap is unused

On Mon, Dec 03, 2012 at 01:56:27AM +0200, Dimitrios Apostolou wrote:
> 
> Please take a look at [2], it's what is called in that case. Via
> this call all slabs that have registered a shrinker, get actually
> shrinked. Other fs do, but I can't find whether ext4 actually
> registers a shrinker for its slabs.

The ext4 inode cache gets shrunk via the VFS layer, in
prune_icache_sb() in fs/inode.c.  This is true for all file systems'
inode caches.

If you are under heavy memory pressure, or if you run echo 3 >
/proc/sys/vm/drop_caches, and the inodes aren't getting dropped, then
it's because the inodes are getting pinned for some reason --- i.e.,
they are referenced via some entry in the dentry cache, perhaps there
are files open, or processes are cd'ed into a directory, etc.

So an example of what happens with the ext4_inode_cache before and
after running "echo 3 > /proc/sys/vm/drop_caches":

ext4_inode_cache  183379 183379   1872   17    8 : tunables    0    0    0 : slabdata  10787  10787      0

ext4_inode_cache    1595   6120   1872   17    8 : tunables    0    0    0 : slabdata    360    360      0

(What's left is due to all of the executable files, shared libraries,
current directories, and open files for all my processes running on my
system --- which for context is Xfce plus Chrome, emacs, mutt, and a
bunch of terminal windows plus the usual assortment of system daemons.)

If you are seeing a large number of inodes still in the ext4 inode
cache after using drop_caches, then I'd look to see whether you have
something like SELinux or auditing enabled which is pinning a bunch of
dentries or inodes, or whether your backup program (or some other
program running on your system) is keeping lots of directories or
inodes open for some reason.

Hope this helps,

					- Ted
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/