linux-kernel - [PATCH 0/3]: compressed in-memory swapping

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <49BF8ABC.6040805@vflare.org>
Date:	Tue, 17 Mar 2009 17:04:20 +0530
From:	Nitin Gupta <ngupta@...are.org>
To:	linux-kernel@...r.kernel.org
Subject: [PATCH 0/3]: compressed in-memory swapping

Hi,

Project home: http://code.google.com/p/compcache/

It allows creating a RAM based block device which acts as swap disk.
Pages swapped to this device are compressed and stored in memory itself.
This is a big win over swapping to slow hard-disk which are typically used
as swap disk. For flash, these suffer from wear-leveling issues when used
as swap disk - so again its helpful. For swapless systems, it allows more
apps to run.

Its now part of Ubuntu, ALT Linux and some other distros.

Some use cases:
  - Embedded Devices: Memory is scarce and adding more memory increases
    device cost. Also, flash storage suffers from wear-levelling issues.
  - Virtualization: For host/hypervisor, VM/guest memory is all anonymous
    memory. So, compcache can compress any part of guest. So we can host
    more VMs on same amount of memory.
  - Thinclients.

Testing/Performance:
  - Testing for use on Thinclients (contrib: Nai Xia):
     Details: http://code.google.com/p/compcache/wiki/LTSPPerf
     Summary: http://code.google.com/p/compcache/wiki/LTSPPerfSummary

     Summary: after the compcache was loaded:
         - The time of paging down one pdf page was reduced to 1/4~1/100
         - The time of switching from one firefox tab to another was reduced to 1/6
         - The capacity of kpdf was be increased from 2 pdf files to 11 pdf files.
         - The capacity of firefox was increased from 6 web pages to 15 web pages.

  - Testing on x86 and x64 VMs (256/512MB RAM, 1 VCPU, 512MB Swap):
    Started Fedora 10 and some "generic desktop" apps (KDE4, Firefox,
    filemanager, editors) - without compcache, system swapped heavily
    and very unresponsive. With compcache, very noticeable improvement
    in responsiveness.

  - Some reported running on PS3 systems though no performance stats
    available here.

Limitations:
  - Being a "swap disk" it can compress only anonymous memory. But as mentioned
    above, at hypervisor/host level, all guest memory is anonymous so any part
    of guest memory can be compressed.
  - Memory allocator used is not scalable at all (work in progress).
  - No run-time defragmentation, swapping of allocated memory - however,
    incompressible pages are forwarded to physical swap and can set limit
    on amount of memory used by compcache.

I do not claim performance gain for any arbitrary workload but as testing
shows, it certainly helps in some cases :)

Also note that this is a low risk change - it does not modify any part
of kernel. Its just an additional module. In case you find it hurting
performance, just unload this module and maybe use normal disk swap.

Any reviews/comments/suggestions are welcome.

Thanks,
Nitin
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/