phc-discussions - Re: [PHC] OMG we have benchmarks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20150402054132.GA18634@openwall.com>
Date: Thu, 2 Apr 2015 08:41:32 +0300
From: Solar Designer <solar@...nwall.com>
To: discussions@...sword-hashing.net
Subject: Re: [PHC] OMG we have benchmarks

On Wed, Apr 01, 2015 at 09:35:27PM -0700, Bill Cox wrote:
> These charts look like Lyra2 is still running with 2 threads,

Maybe or maybe not.  Milan wrote that "Lyra2 should be for 1 thread
(-DnPARALLEL=1)".  I think this should be verified by watching "top"
while it's running.

> and Yescrypt
> is still running with 6 rounds rather than a more comparable 2 rounds.

Yes, but that's how it should be on the main comparison charts, because
these are the default settings of the current PHC submissions.

Milan may produce additional comparisons focused on specific use cases,
with tuned settings, but for the main comparisons I think it's fair to
use the defaults (and we have to consider differences other than memory
filling speed with default settings separately).

> These are pretty pictures, but the benchmarks I ran with a bit more care
> showed a very different picture.  I'd hate to see a winner selected based
> on nice coloring and one algorithm running 2 threads vs all the rest with
> 1, and one algorithms running 6 rounds rather than 2.

I'd also be unhappy if the panel doesn't consider these other differences.
(I'm not sure about the 1 vs. 2 threads, though - it might be that Lyra2
is actually running 1 thread here.)

Alexander