linux-kernel - Problem with RSA test from testmgr

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170228155953.GA1732@Red>
Date:   Tue, 28 Feb 2017 16:59:53 +0100
From:   Corentin Labbe <clabbe.montjoie@...il.com>
To:     herbert@...dor.apana.org.au, linux-crypto@...r.kernel.org
Cc:     linux-kernel@...r.kernel.org
Subject: Problem with RSA test from testmgr

hello

I work on the sun8i-ce crypto accelerator and I have some problem with the RSA part.

The RSA register fail at the first RSA test (encrypt 512bit) with this output:
[ 8480.146843] alg: akcipher: encrypt test failed. Invalid output
[ 8480.146871] 00000000: 6e 7c 8a 75 e7 30 80 d1 5e ab 9b db a2 cf ed db
[ 8480.146897] 00000010: c9 b2 db 43 bd 9a b9 75 27 f3 73 d9 73 b7 81 8c
[ 8480.146921] 00000020: 49 e8 45 fc 43 44 f5 6d f0 f7 b8 f2 ae 6b ae 49
[ 8480.146946] 00000030: 1b 8e 50 c6 88 4e 99 09 78 14 f2 5d 99 c3 7f f9
[ 8480.146995] alg: akcipher: test 1 failed for rsa-sun8i-ce, err=-22

But with the same parameters (msg, n, e) openssl give me exactly this output.

So what I miss for made it work ?
In which format testmgr expect the output data ?

Thanks
Regards
Corentin Labbe