linux-kernel - Re: af_alg01.c:36: TFAIL: instantiated nested hmac algorithm ('hmac(hmac(md5))')!

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20240924222839.GC1585@sol.localdomain>
Date: Tue, 24 Sep 2024 15:28:39 -0700
From: Eric Biggers <ebiggers@...nel.org>
To: Herbert Xu <herbert@...dor.apana.org.au>
Cc: Naresh Kamboju <naresh.kamboju@...aro.org>,
	LTP List <ltp@...ts.linux.it>,
	open list <linux-kernel@...r.kernel.org>,
	Linux Crypto Mailing List <linux-crypto@...r.kernel.org>,
	lkft-triage@...ts.linaro.org,
	"David S. Miller" <davem@...emloft.net>,
	Anders Roxell <anders.roxell@...aro.org>,
	Dan Carpenter <dan.carpenter@...aro.org>,
	Arnd Bergmann <arnd@...db.de>, chrubis <chrubis@...e.cz>
Subject: Re: af_alg01.c:36: TFAIL: instantiated nested hmac algorithm
 ('hmac(hmac(md5))')!

On Tue, Sep 24, 2024 at 09:30:39PM +0800, Herbert Xu wrote:
> On Tue, Sep 24, 2024 at 03:16:09PM +0530, Naresh Kamboju wrote:
> >
> > Warning log:
> > ----------
> > tst_test.c:1617: TINFO: Timeout per run is 0h 02m 30s
> > af_alg01.c:36: TFAIL: instantiated nested hmac algorithm ('hmac(hmac(md5))')!
> > tst_af_alg.c:46: TBROK: unexpected error binding AF_ALG socket to hash
> > algorithm 'hmac(hmac(md5))': EINVAL (22)
> 
> This is expected.  You need to fix the test to not treat this as an
> error.
> 

So the error code changed from ENOENT to EINVAL.  These particular LTP tests
(af_alg01 and af_alg03) could check for either ENOENT or EINVAL, and it would
preserve the main point of the tests.

I do feel that the previous error code, ENOENT, was more logical though.  From
userspace's perspective, these invalid algorithms don't exist and don't need to
be treated any different from other algorithms that don't exist.

Has it been checked what else in userspace might need to be updated as a result
of this change?

Looking at libkcapi for example
(https://github.com/smuellerDD/libkcapi/blob/master/lib/kcapi.h#L125) it passes
the error code up and documents it as part of its API:

     * @return 0 upon success;
     *         -ENOENT - algorithm not available;
     *         -EOPNOTSUPP - AF_ALG family not available;
     *         -EINVAL - accept syscall failed
     *         -ENOMEM - cipher handle cannot be allocated
     */
    int kcapi_cipher_init(struct kcapi_handle **handle, const char *ciphername,
                          uint32_t flags);

So that is now wrong, as "algorithm not available" can now be either ENOENT or
EINVAL.

I don't really see the reason for this churn.  The kernel commit which made this
change (795f85fca229) provides no explanation either.

- Eric