linux-kernel - [PATCH] ipc/sem.c: prevent ENOMEM in semop() w/ SEM

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <501BC8BE.6000405@jp.fujitsu.com>
Date:	Fri, 03 Aug 2012 21:49:02 +0900
From:	Seiichi Ikarashi <s.ikarashi@...fujitsu.com>
To:	manfred@...orfullife.com, linux-kernel@...r.kernel.org
Subject: [PATCH] ipc/sem.c: prevent ENOMEM in semop() w/ SEM_UNDO flag

semop() with SEM_UNDO sem_flg can result in ENOMEM even after
succeeding semget() with large nsems. This is because
semop() uses kzalloc() via find_alloc_undo() though
semget() uses vmalloc() via ipc_rcu_alloc().
This patch makes semop() be able to use vmalloc() via ipc_alloc().

Signed-off-by: Seiichi Ikarashi <s.ikarashi@...fujitsu.com>

--- a/ipc/sem.c	2012-08-03 16:52:01.000000000 +0900
+++ b/ipc/sem.c	2012-08-03 20:40:57.000000000 +0900
@@ -1258,11 +1258,12 @@ static struct sem_undo *find_alloc_undo(
 	sem_getref_and_unlock(sma);
 
 	/* step 2: allocate new undo structure */
-	new = kzalloc(sizeof(struct sem_undo) + sizeof(short)*nsems, GFP_KERNEL);
+	new = ipc_alloc(sizeof(struct sem_undo) + sizeof(short)*nsems, GFP_KERNEL);
 	if (!new) {
 		sem_putref(sma);
 		return ERR_PTR(-ENOMEM);
 	}
+	memset(new, 0, sizeof(struct sem_undo) + sizeof(short)*nsems);
 
 	/* step 3: Acquire the lock on semaphore array */
 	sem_lock_and_putref(sma);
@@ -1348,7 +1349,7 @@ SYSCALL_DEFINE4(semtimedop, int, semid, 
 	if (nsops > ns->sc_semopm)
 		return -E2BIG;
 	if(nsops > SEMOPM_FAST) {
-		sops = kmalloc(sizeof(*sops)*nsops,GFP_KERNEL);
+		sops = ipc_alloc(sizeof(*sops)*nsops,GFP_KERNEL);
 		if(sops==NULL)
 			return -ENOMEM;
 	}
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/