linux-kernel - [PATCH 2/5] lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <1464297595-24032-3-git-send-email-nicstange@gmail.com>
Date:	Thu, 26 May 2016 23:19:52 +0200
From:	Nicolai Stange <nicstange@...il.com>
To:	Herbert Xu <herbert@...dor.apana.org.au>
Cc:	David Howells <dhowells@...hat.com>,
	Tadeusz Struk <tadeusz.struk@...el.com>,
	Michal Marek <mmarek@...e.com>, linux-crypto@...r.kernel.org,
	linux-kernel@...r.kernel.org, Nicolai Stange <nicstange@...il.com>
Subject: [PATCH 2/5] lib/digsig: digsig_verify_rsa(): return -EINVAL if modulo length is zero

Currently, if digsig_verify_rsa() detects that the modulo's length is zero,
i.e. mlen == 0, it returns -ENOMEM which doesn't really fit here.

Make digsig_verify_rsa() return -EINVAL upon mlen == 0.

Signed-off-by: Nicolai Stange <nicstange@...il.com>
---
 lib/digsig.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)

diff --git a/lib/digsig.c b/lib/digsig.c
index a121cbc..55b8b2f 100644
--- a/lib/digsig.c
+++ b/lib/digsig.c
@@ -114,13 +114,15 @@ static int digsig_verify_rsa(struct key *key,
 		datap += remaining;
 	}
 
-	err = -ENOMEM;
-
 	mblen = mpi_get_nbits(pkey[0]);
 	mlen = DIV_ROUND_UP(mblen, 8);
 
-	if (mlen == 0)
+	if (mlen == 0) {
+		err = -EINVAL;
 		goto err;
+	}
+
+	err = -ENOMEM;
 
 	out1 = kzalloc(mlen, GFP_KERNEL);
 	if (!out1)
-- 
2.8.2