Date: Wed, 11 Feb 2015 22:24:07 +0100
From: Jean-Philippe Aumasson <jeanphilippe.aumasson@...il.com>
To: discussions@...sword-hashing.net
Subject: Re: [PHC] PHC status report

Succinct response (not much time to elaborate):

* You will find "secretive selection" and non-public discussions of
the committee panel in all crypto competitions: AES, NESSIE, eSTREAM,
SHA-3, CAESAR, and PHC.

* A non-negligible difference of PHC compared to other projects: we're
all unpaid volunteers, working on this project on our free time, where
NIST had a dedicated team for AES and SHA-3, a budget to organize
workshops, etc.

* We wished we had time and resources to prepare a lengthy and
detailed report, but that wasn't the case. We felt that it was better
to publish such a summary than nothing at all.


On Wed, Feb 11, 2015 at 10:14 PM, Somitra Sanadhya <somitra@...td.ac.in> wrote:
> Dear all,
>
> I think the discussion is not leading to a meaningful and factual direction.
> Let me humbly try to steer the discussion towards concrete measures and stop
> this blame game. There are some TO-DO items in the comments, which can help
> restore the credibility of the competition (at least among some of us who
> are not happy about it).
>
> 1. The panel should make the voting and discussion on the selection criteria
> public.
>
> I refer to the AES competition archived selection criteria here:
> http://csrc.nist.gov/archive/aes/pre-round1/aes_9709.htm Note the language
> from section 4: "In order to provide a basis for the analysis and evaluation
> of encryption algorithms submitted to be considered for incorporation into
> the FIPS for AES, evaluation criteria will be used to review candidate
> algorithms.  All of NIST’s analysis results will be made publicly
> available."
>
> It is in the spirit of an open competition to keep the process transparent.
> All discussion leading to the selection should have been public already.
>
> 2. I had pointed this earlier that the comment on our design Rig is as
> follows: "Similar to Catena, but received less attention (cf. bugs found in
> the specification and code)".
>
> There are two parts to this comment. (a) Similar to Catena. (b) bugs found.
>
> I would have liked the panel to clarify each of the two points with respect
> to the revised submission Rig v2 which is the one the PHC website lists
> since October 2014. Clearly, (b) is false for this. Further, is (a)
> measurable by any means ?  Aren't there novel ideas in the design of Rig,
> apart from ideas from Catena ? Given the publicly available eprint report
> which I referred to in my previous mail, is it fair to dismiss the design in
> this single sentence ? If Rig v2 was not being considered, shouldn't the
> designers have been informed at the time of the revised submission itself
> that this version will not be evaluated any more ? (Related: weren't other
> design also allowed to be modified around the same time ? The changes were
> not overhauling the design, these were minor modification to handle the
> issues which Bill Cox found. I don't think there was any comment on the
> design after that. If the panel was not looking at the revised submission
> then we could have as well saved our time to do other things, rather than
> investing our time in something which no one was interested in looking at.)
>
> 3. It is not just our design. Most designs have one line comments on them in
> the document shared earlier. To say that the panel could not prepare a
> detailed document is mocking the competition. As pointed by Krisztian
> earlier, many of these one liners are actually not factual but based on
> opinions. The report should have had meaningful comparison of the
> candidates, not just one liners on the entries. Dismissing entries by such
> one liners is devaluing the effort put by so many designers in the
> competition.
>
> If you want some specific metrics, then here is a randomly thought list
> which is not exhaustive: performance on various platforms, cryptographic
> strength, memory hardness, .... (add whatever else you like, make a baseline
> and compare all entries on some rational basis).
>
> In my humble opinion, the bitterness which we are witnessing in the mailing
> list is due to the secretive selection and the improper rationale for
> selection in the document. If these were public and based on detailed
> discussions, I don't think anyone would have complained. IMHO, the panel
> members should have already realized that there is a lot to blame themselves
> rather than the people questioning their decision now. To blame the
> questions on the "frustration of non-finalists" is not showing the maturity
> expected from a panel, which contains many good people whom many of us
> trusted (if that was not the case then you wouldn't have received so many
> submissions in the first place). Honestly, please discuss with some
> researchers in universities around you about the way the selection has
> happened so far, showing them the "selection rationale document" and "the
> process followed" (the secret voting, and not even following that voting
> perfectly;  claiming that this was in addition to the private discussion
> etc). I am quite certain that none of them will favor the process as
> followed. All of it was easily avoided by keeping the process in public
> domain and having a well thought out selection document.
>
> To quote Bruce Schneier on the AES competition (a losing finalist for the
> competition): "I have nothing but good things to say about NIST and the AES
> process".
>
> We were expecting a competition in the spirit of AES, but unfortunately
> things haven't gone that way.
>
> 4. I deviate slightly to the "flexibility" and "simplicity" ideas of the AES
> competition. The AES competition had a criteria termed "simplicity", but it
> already created lots of discussion/confusion that time. To quote a few lines
> from the TwoFish team (https://www.schneier.com/paper-twofish-final.pdf) "
> "Simplicity” is the NIST criterion that’s hardest to describe. Lines of
> pseudocode, number of mathematical equations, density of lines in a block
> diagram: these are all potential measures of simplicity. Our worry about
> simplicity as a measure is that the simplest algorithms are often the
> easiest to break. ...".
>
>  If things are hazy then people will question them.
>
> Regards.
> Somitra
>

Powered by blists - more mailing lists