linux-kernel - Re: Multi Core Support for compression in compression.c

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <53D5BBCA.3020109@gmail.com>
Date:	Sun, 27 Jul 2014 22:56:10 -0400
From:	Austin S Hemmelgarn <ahferroin7@...il.com>
To:	Nick Krause <xerofoify@...il.com>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
	"linux-btrfs@...r.kernel.org SYSTEM list:BTRFS FILE" 
	<linux-btrfs@...r.kernel.org>
Subject: Re: Multi Core Support for compression in compression.c

On 07/27/2014 04:47 PM, Nick Krause wrote:
> This may be a bad idea , but compression in brtfs seems to be only
> using one core to compress.
> Depending on the CPU used and the amount of cores in the CPU we can
> make this much faster
> with multiple cores. This seems bad by my reading at least I would
> recommend for writing compression
> we write a function to use a certain amount of cores based on the load
> of the system's CPU not using
> more then 75% of the system's CPU resources as my system when idle has
> never needed more
> then one core of my i5 2500k to run when with interrupts for opening
> eclipse are running. For reading
> compression on good core seems fine to me as testing other compression
> software for reads , it's
> way less CPU intensive.
> Cheers Nick
We would probably get a bigger benefit from taking an approach like
SquashFS has recently added, that is, allowing multi-threaded
decompression fro reads, and decompressing directly into the pagecache.
 Such an approach would likely make zlib compression much more scalable
on large systems.



Download attachment "smime.p7s" of type "application/pkcs7-signature" (2967 bytes)