linux-kernel - Re: [PATCH] Re: 2.6.21-rc7-mm2 -- hvsi console driver registration failure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 4 May 2007 13:09:52 -0700
From:	Andrew Morton <akpm@...ux-foundation.org>
To:	Andy Whitcroft <apw@...dowen.org>
Cc:	gregkh@...e.de, paulus@...ba.org, linux-kernel@...r.kernel.org,
	linuxppc-dev@...abs.org, Christoph Lameter <clameter@...r.sgi.com>
Subject: Re: [PATCH] Re: 2.6.21-rc7-mm2 -- hvsi console driver registration
 failure

On Fri, 04 May 2007 12:38:58 +0100
Andy Whitcroft <apw@...dowen.org> wrote:

> 
> Trying to get 2.6.21-rc7-mm2 to boot on large PPC64 seems to be a
> bit of a challenge.  We have been seeing panics on boot from the
> hvsi driver:
> 
> 	Couldn't register hvsi console driver
> 
> Tracking this back, this seems to come from hvsi driver trying to
> register itself via tty_register_driver() with a zero units.
> 
> The failure is triggered by a change in semantics for kmalloc()
> between SLAB and SLUB; kmalloc(0) now returns NULL rather than an
> allocation at the smallest size.  Looking at the code in question
> even when the allocation succeeds we will not actually use the
> memory when device->num is zero.

OK, thanks for working that out.

Christoph, we should be emitting loud warnings so that this problem is easy
to debug.

Better, we should be emitting loud warnigns which then disable themselves
and then succeeding the allocation so that people can proceed with their
kernel testing.

When all the loud-warning sites have been fixed, we can take that code out
again.

The present situation is maximally tester-hostile.

> It is not clear to me if this is a bug in the hvsi driver in that
> it should specify some units.  It seems we will try and reserve zero
> devices in this case, which seems pointless.
> 
> I have tested with the patch below which seems safe to me and stops
> the errors and even seems to make the console work.  But perhaps
> someone with more driver fu, could verify if driver->num of zero
> has any meaning and kick this to the hvsi people if not.
> 
> -apw
> 
> === 8< ===
> tty_register_driver: only allocate tty instances when defined
> 
> If device->num is zero we attempt to kmalloc() zero bytes.
> When SLUB is enabled this returns a null pointer and take that as
> an allocation failure and fail the device register.  Check for no
> devices and avoid the allocation.
> 
> Signed-off-by: Andy Whitcroft <apw@...dowen.org>
> ---
> diff --git a/drivers/char/tty_io.c b/drivers/char/tty_io.c
> index 959a616..71c4579 100644
> --- a/drivers/char/tty_io.c
> +++ b/drivers/char/tty_io.c
> @@ -3724,7 +3724,7 @@ int tty_register_driver(struct tty_driver *driver)
>  	if (driver->flags & TTY_DRIVER_INSTALLED)
>  		return 0;
>  
> -	if (!(driver->flags & TTY_DRIVER_DEVPTS_MEM)) {
> +	if (!(driver->flags & TTY_DRIVER_DEVPTS_MEM) && driver->num) {
>  		p = kmalloc(driver->num * 3 * sizeof(void *), GFP_KERNEL);
>  		if (!p)
>  			return -ENOMEM;
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/