linux-kernel - Re: [RFC][PATCH] update /proc/sys/vm/drop

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20100916103326.026ca03f.kamezawa.hiroyu@jp.fujitsu.com>
Date:	Thu, 16 Sep 2010 10:33:26 +0900
From:	KAMEZAWA Hiroyuki <kamezawa.hiroyu@...fujitsu.com>
To:	Dave Hansen <dave@...ux.vnet.ibm.com>
Cc:	Tim Pepper <lnxninja@...ux.vnet.ibm.com>,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org,
	linux-fsdevel <linux-fsdevel@...r.kernel.org>
Subject: Re: [RFC][PATCH] update /proc/sys/vm/drop_caches documentation

On Wed, 15 Sep 2010 18:21:47 -0700
Dave Hansen <dave@...ux.vnet.ibm.com> wrote:

> On Thu, 2010-09-16 at 09:12 +0900, KAMEZAWA Hiroyuki wrote:
> > I hear a customer's case. His server generates 3-80000+ new dentries per day
> > and dentries will be piled up to 1000000+ in a month. This makes open()'s 
> > performance very bad because Hash-lookup will be heavy. (He has very big memory.)
> > 
> > What we could ask him was
> >   - rewrite your application. or
> >   - reboot once in a month (and change hash size) or
> >   - drop_cache once in a month
> > 
> > Because their servers cannot stop, he used drop_caches once in a month
> > while his server is idle, at night. Changing HashSize cannot be a permanent
> > fix because he may not stop the server for years.
> 
> That is a really interesting case.
> 
> They must have a *ton* of completely extra memory laying around.  Do
> they not have much page cache activity? 

I hear they have a ton of extra memory. Just open() slows down.

> It usually balances out the  dentry/inode caches.
> 
> Would this user be better off with a smaller dentry hash in general? 

Maybe. I hear most of files were created-but-never-used data and logs.

> Is it special hardware that should _have_ a lower default hash size?
> 

I'm not sure. I think they have no boot option of hash size.


> > For rare users who have 10000000+ of files and tons of free memory, drop_cache
> > can be an emergency help. 
> 
> In this case, though, would a WARN_ON() in an emergency be such a bad
> thing?  They evidently know what they're doing, and shouldn't be put off
> by it.
> 

Showing "Warning" means ", it's possibly bug." for almost all customers.
We'll get tons of "regression" report ;)

If you really want to add messages, please raise log level.
NOTICE or INFO sounds better(and moderate) to me because it's easy to explain
"Don't worry about the message, your kernel is stable and don't need to reboot.
 But please check the peformance, it tends to go bad. You lose cache.".
BTW, what(1or2or3) was writtern to "drop_cache" is important. please show.

Thanks,
-Kame







--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/