Date: Wed, 11 Feb 2015 23:31:08 +0200
From: Somitra Sanadhya <somitra@...td.ac.in>
To: discussions@...sword-hashing.net
Subject: Re: [PHC] PHC status report

In all academic paper submissions in conferences, researchers submit their
papers and "unpaid volunteers" review the papers. Usually the reviews the
submitters receive are quite long and discuss the papers maturely. (In
addition, the reviewers have some additional discussion, which is not
shared with the submitter and that is perfectly acceptable). But missing
the detailed reviews themselves is strange, at least to me. We weren't
expecting workshops in the style of AES/SHA-3 anyway.

Regards.


On Wed, Feb 11, 2015 at 11:24 PM, Jean-Philippe Aumasson <
jeanphilippe.aumasson@...il.com> wrote:

> Succinct response (not much time to elaborate):
>
> * You will find "secretive selection" and non-public discussions of
> the committee panel in all crypto competitions: AES, NESSIE, eSTREAM,
> SHA-3, CAESAR, and PHC.
>
> * A non-negligible difference of PHC compared to other projects: we're
> all unpaid volunteers, working on this project on our free time, where
> NIST had a dedicated team for AES and SHA-3, a budget to organize
> workshops, etc.
>
> * We wished we had time and resources to prepare a lengthy and
> detailed report, but that wasn't the case. We felt that it was better
> to publish such a summary than nothing at all.
>
>
> On Wed, Feb 11, 2015 at 10:14 PM, Somitra Sanadhya <somitra@...td.ac.in>
> wrote:
> > Dear all,
> >
> > I think the discussion is not leading to a meaningful and factual
> direction.
> > Let me humbly try to steer the discussion towards concrete measures and
> stop
> > this blame game. There are some TO-DO items in the comments, which can
> help
> > restore the credibility of the competition (at least among some of us who
> > are not happy about it).
> >
> > 1. The panel should make the voting and discussion on the selection
> criteria
> > public.
> >
> > I refer to the AES competition archived selection criteria here:
> > http://csrc.nist.gov/archive/aes/pre-round1/aes_9709.htm Note the
> language
> > from section 4: "In order to provide a basis for the analysis and
> evaluation
> > of encryption algorithms submitted to be considered for incorporation
> into
> > the FIPS for AES, evaluation criteria will be used to review candidate
> > algorithms.  All of NIST’s analysis results will be made publicly
> > available."
> >
> > It is in the spirit of an open competition to keep the process
> transparent.
> > All discussion leading to the selection should have been public already.
> >
> > 2. I had pointed this earlier that the comment on our design Rig is as
> > follows: "Similar to Catena, but received less attention (cf. bugs found
> in
> > the specification and code)".
> >
> > There are two parts to this comment. (a) Similar to Catena. (b) bugs
> found.
> >
> > I would have liked the panel to clarify each of the two points with
> respect
> > to the revised submission Rig v2 which is the one the PHC website lists
> > since October 2014. Clearly, (b) is false for this. Further, is (a)
> > measurable by any means ?  Aren't there novel ideas in the design of Rig,
> > apart from ideas from Catena ? Given the publicly available eprint report
> > which I referred to in my previous mail, is it fair to dismiss the
> design in
> > this single sentence ? If Rig v2 was not being considered, shouldn't the
> > designers have been informed at the time of the revised submission itself
> > that this version will not be evaluated any more ? (Related: weren't
> other
> > design also allowed to be modified around the same time ? The changes
> were
> > not overhauling the design, these were minor modification to handle the
> > issues which Bill Cox found. I don't think there was any comment on the
> > design after that. If the panel was not looking at the revised submission
> > then we could have as well saved our time to do other things, rather than
> > investing our time in something which no one was interested in looking
> at.)
> >
> > 3. It is not just our design. Most designs have one line comments on
> them in
> > the document shared earlier. To say that the panel could not prepare a
> > detailed document is mocking the competition. As pointed by Krisztian
> > earlier, many of these one liners are actually not factual but based on
> > opinions. The report should have had meaningful comparison of the
> > candidates, not just one liners on the entries. Dismissing entries by
> such
> > one liners is devaluing the effort put by so many designers in the
> > competition.
> >
> > If you want some specific metrics, then here is a randomly thought list
> > which is not exhaustive: performance on various platforms, cryptographic
> > strength, memory hardness, .... (add whatever else you like, make a
> baseline
> > and compare all entries on some rational basis).
> >
> > In my humble opinion, the bitterness which we are witnessing in the
> mailing
> > list is due to the secretive selection and the improper rationale for
> > selection in the document. If these were public and based on detailed
> > discussions, I don't think anyone would have complained. IMHO, the panel
> > members should have already realized that there is a lot to blame
> themselves
> > rather than the people questioning their decision now. To blame the
> > questions on the "frustration of non-finalists" is not showing the
> maturity
> > expected from a panel, which contains many good people whom many of us
> > trusted (if that was not the case then you wouldn't have received so many
> > submissions in the first place). Honestly, please discuss with some
> > researchers in universities around you about the way the selection has
> > happened so far, showing them the "selection rationale document" and "the
> > process followed" (the secret voting, and not even following that voting
> > perfectly;  claiming that this was in addition to the private discussion
> > etc). I am quite certain that none of them will favor the process as
> > followed. All of it was easily avoided by keeping the process in public
> > domain and having a well thought out selection document.
> >
> > To quote Bruce Schneier on the AES competition (a losing finalist for the
> > competition): "I have nothing but good things to say about NIST and the
> AES
> > process".
> >
> > We were expecting a competition in the spirit of AES, but unfortunately
> > things haven't gone that way.
> >
> > 4. I deviate slightly to the "flexibility" and "simplicity" ideas of the
> AES
> > competition. The AES competition had a criteria termed "simplicity", but
> it
> > already created lots of discussion/confusion that time. To quote a few
> lines
> > from the TwoFish team (https://www.schneier.com/paper-twofish-final.pdf)
> "
> > "Simplicity” is the NIST criterion that’s hardest to describe. Lines of
> > pseudocode, number of mathematical equations, density of lines in a block
> > diagram: these are all potential measures of simplicity. Our worry about
> > simplicity as a measure is that the simplest algorithms are often the
> > easiest to break. ...".
> >
> >  If things are hazy then people will question them.
> >
> > Regards.
> > Somitra
> >
>

Content of type "text/html" skipped

Powered by blists - more mailing lists