linux-kernel - Re: [PATCH] Re: 2.6.21-rc7-mm2 -- hvsi console driver registration failure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 4 May 2007 14:04:08 -0500
From:	linas@...tin.ibm.com (Linas Vepstas)
To:	Andy Whitcroft <apw@...dowen.org>
Cc:	gregkh@...e.de, paulus@...ba.org, Andrew Morton <akpm@...l.org>,
	linuxppc-dev@...abs.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] Re: 2.6.21-rc7-mm2 -- hvsi console driver registration failure

On Fri, May 04, 2007 at 12:38:58PM +0100, Andy Whitcroft wrote:
> 
> Trying to get 2.6.21-rc7-mm2 to boot on large PPC64 seems to be a
> bit of a challenge.  We have been seeing panics on boot from the
> hvsi driver:
> 
> 	Couldn't register hvsi console driver
> 
> Tracking this back, this seems to come from hvsi driver trying to
> register itself via tty_register_driver() with a zero units.
> 
> The failure is triggered by a change in semantics for kmalloc()
> between SLAB and SLUB; kmalloc(0) now returns NULL rather than an
> allocation at the smallest size.  Looking at the code in question
> even when the allocation succeeds we will not actually use the
> memory when device->num is zero.
> 
> It is not clear to me if this is a bug in the hvsi driver in that
> it should specify some units.  It seems we will try and reserve zero
> devices in this case, which seems pointless.

Yes, it seems pointless to me ... 

> I have tested with the patch below which seems safe to me and stops
> the errors and even seems to make the console work.  But perhaps
> someone with more driver fu, could verify if driver->num of zero
> has any meaning and kick this to the hvsi people if not.

Hollis nominated me to be "hvsi people", although I'm near-totally
ignorant of the thing.

If hvsi_count is zero, then the device tree did not have any
"serial" nodes that speak "hvterm-protocol". The hvsi should not 
have even tried to register anything. The attached patch seems more 
to the point.

--linas


The hvsi driver is used whenever the device-tree contains
nodes for serial ports, and those serial ports speak the hvterm
protocol. However, if no such nodes are found, then the hvsi
driver should not even register. 

This patch avoids a kernel panic with "Couldn't register hvsi 
console driver". 

In addition, this patch makes tty_register_driver refuse
to do anything, if there are no actual tty ports to be 
registered.

Utterly & completely untested.

Signed-off-by: Linas Vepstas <linas@...tin.ibm.com>

----
 drivers/char/hvsi.c   |    4 ++++
 drivers/char/tty_io.c |    3 +++
 2 files changed, 7 insertions(+)

Index: linux-2.6.21-rc7-mm2/drivers/char/hvsi.c
===================================================================
--- linux-2.6.21-rc7-mm2.orig/drivers/char/hvsi.c	2007-04-26 15:37:33.000000000 -0500
+++ linux-2.6.21-rc7-mm2/drivers/char/hvsi.c	2007-05-04 13:55:56.000000000 -0500
@@ -1148,6 +1148,10 @@ static int __init hvsi_init(void)
 {
 	int i;
 
+	/* No serial hvterm-protocol device-tree nodes found. */
+	if (hvsi_count == 0)
+		return 0;
+
 	hvsi_driver = alloc_tty_driver(hvsi_count);
 	if (!hvsi_driver)
 		return -ENOMEM;
Index: linux-2.6.21-rc7-mm2/drivers/char/tty_io.c
===================================================================
--- linux-2.6.21-rc7-mm2.orig/drivers/char/tty_io.c	2007-04-26 15:37:33.000000000 -0500
+++ linux-2.6.21-rc7-mm2/drivers/char/tty_io.c	2007-05-04 13:54:14.000000000 -0500
@@ -3724,6 +3724,9 @@ int tty_register_driver(struct tty_drive
 	if (driver->flags & TTY_DRIVER_INSTALLED)
 		return 0;
 
+	if (driver->num == 0)
+		return -ENODEV;
+
 	if (!(driver->flags & TTY_DRIVER_DEVPTS_MEM)) {
 		p = kmalloc(driver->num * 3 * sizeof(void *), GFP_KERNEL);
 		if (!p)
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/