lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <4fe4528cb20beb569a8f2ac817c776e5@pinky>
Date:	Fri, 04 May 2007 12:38:58 +0100
From:	Andy Whitcroft <apw@...dowen.org>
To:	gregkh@...e.de, paulus@...ba.org
Cc:	Andrew Morton <akpm@...l.org>, Andy Whitcroft <apw@...dowen.org>,
	linux-kernel@...r.kernel.org, linuxppc-dev@...abs.org
Subject: [PATCH] Re: 2.6.21-rc7-mm2 -- hvsi console driver registration failure


Trying to get 2.6.21-rc7-mm2 to boot on large PPC64 seems to be a
bit of a challenge.  We have been seeing panics on boot from the
hvsi driver:

	Couldn't register hvsi console driver

Tracking this back, this seems to come from hvsi driver trying to
register itself via tty_register_driver() with a zero units.

The failure is triggered by a change in semantics for kmalloc()
between SLAB and SLUB; kmalloc(0) now returns NULL rather than an
allocation at the smallest size.  Looking at the code in question
even when the allocation succeeds we will not actually use the
memory when device->num is zero.

It is not clear to me if this is a bug in the hvsi driver in that
it should specify some units.  It seems we will try and reserve zero
devices in this case, which seems pointless.

I have tested with the patch below which seems safe to me and stops
the errors and even seems to make the console work.  But perhaps
someone with more driver fu, could verify if driver->num of zero
has any meaning and kick this to the hvsi people if not.

-apw

=== 8< ===
tty_register_driver: only allocate tty instances when defined

If device->num is zero we attempt to kmalloc() zero bytes.
When SLUB is enabled this returns a null pointer and take that as
an allocation failure and fail the device register.  Check for no
devices and avoid the allocation.

Signed-off-by: Andy Whitcroft <apw@...dowen.org>
---
diff --git a/drivers/char/tty_io.c b/drivers/char/tty_io.c
index 959a616..71c4579 100644
--- a/drivers/char/tty_io.c
+++ b/drivers/char/tty_io.c
@@ -3724,7 +3724,7 @@ int tty_register_driver(struct tty_driver *driver)
 	if (driver->flags & TTY_DRIVER_INSTALLED)
 		return 0;
 
-	if (!(driver->flags & TTY_DRIVER_DEVPTS_MEM)) {
+	if (!(driver->flags & TTY_DRIVER_DEVPTS_MEM) && driver->num) {
 		p = kmalloc(driver->num * 3 * sizeof(void *), GFP_KERNEL);
 		if (!p)
 			return -ENOMEM;
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ