linux-kernel - RE: cleancache can lead to serious performance degradation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 26 Aug 2011 07:42:32 -0700 (PDT)
From:	Dan Magenheimer <dan.magenheimer@...cle.com>
To:	Seth Jennings <sjenning@...ux.vnet.ibm.com>
Cc:	Nebojsa Trpkovic <trx.lists@...il.com>,
	linux-kernel@...r.kernel.org, Konrad Wilk <konrad.wilk@...cle.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Nitin Gupta <ngupta@...are.org>
Subject: RE: cleancache can lead to serious performance degradation

> From: Seth Jennings [mailto:sjenning@...ux.vnet.ibm.com]
> 
> On 08/25/2011 11:56 AM, Dan Magenheimer wrote:
> 
> > Third, zcache is relatively new and can certainly benefit from
> > the input of other developers.  The lzo1x compression in the kernel
> > is fairly slow; Seth Jennings (cc'ed) is looking into alternate
> > compression technologies.  Perhaps there is a better compression
> > choice more suitable for older-slower processors, probably with a
> > poorer compression ratio.  Further, zcache currently does compression
> > and decompression with interrupts disabled, which may be a
> > significant factor in the slowdowns you've observed.  This should
> > be fixable.
> 
> This was something I've meaning to ask about.  Why are compression
> and decompression done with interrupts disabled?

The "hb->lock" is held during most tmem operations.  If a tmem
operation (or the callback to zcache) is interrupted and the
current cpu is scheduled to run another task, and the new task
calls into tmem, deadlock could occur.

In some cases, I think disabling preemption or bottom-halves instead
of disabling interrupts may be sufficient, but I ran into problems
when I tried that and never got back to it.  Note though that
interrupts are already disabled in some cases when cleancache is
called, so that would only solve part of the problem.

> What would need to change so that we don't have to disable interrupts?

Not easy, but not terribly hard I think:

1) On put, copy the page into tmem uncompressed, and keep a list
of not-yet-compressed pages.  (The copy would still need to be
done with interrupts disabled but copying a page is presumable
one-to-three orders of magnitude faster than compress/decompress.)
2) An independent lower-priority thread would be launched periodically
to compress one or more pages on the uncompressed list and
atomically replace the uncompressed page with the compressed version,
and change all pointers.  Then the uncompressed page could be freed.
There are likely some ugly race conditions in here.
3) On get, ensure not-yet-compressed pages are properly handled.

Dan

P.S. On vacation today, no more email until next week.

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/