linux-kernel - RE: [PATCH] Software Suspend: Fix suspend when console is in VT_AUTO/KD

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <DBC923D919FAB345973A94E6E194E8490438088E@yvrmail01.itc.intrinsyc.com>
Date:	Thu, 8 Mar 2007 21:01:22 -0800
From:	"Andrew Johnson" <ajohnson@...rinsyc.com>
To:	"Daniel Drake" <dsd@...too.org>
Cc:	<pavel@...e.cz>, <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH] Software Suspend: Fix suspend when console is in VT_AUTO/KD_GRAPHICS mode

Daniel Drake wrote:
> Andrew Johnson wrote:
> > When the console is in VT_AUTO/KD_GRAPHICS mode, switching to the
> > SUSPEND_CONSOLE fails, resulting in vt_waitactive() waiting
indefinately
> > or until the task is interrupted.  The following patch tests if a
> > console switch can occur in set_console() and returns early if a
console
> > switch is not possible.
> >
> > Signed-off-by: Andrew Johnson <ajohnson@...rinsyc.com>
> >
> > diff -rup linux-2.6.20.1/drivers/char/vt.c linux/drivers/char/vt.c
> > --- linux-2.6.20.1/drivers/char/vt.c	2007-02-19
22:34:32.000000000 -
> 0800
> > +++ linux/drivers/char/vt.c	2007-03-08 14:15:41.000000000 -0800
> > @@ -2188,10 +2188,20 @@ static void console_callback(struct work
> >  	release_console_sem();
> >  }
> >
> > -void set_console(int nr)
> > +extern char vt_dont_switch;
> > +
> > +int set_console(int nr)
> >  {
> > +	struct vc_data *vc = vc_cons[fg_console].d;
> > +
> > +	if(!vc_cons_allocated(nr) || vt_dont_switch || vc->vc_mode ==
> > KD_GRAPHICS) {
> > +		return -EINVAL;
> > +	}
> > +
> >  	want_console = nr;
> >  	schedule_console_callback();
> > +
> > +	return 0;
> >  }
> 
> I haven't tested, but I think the above -EINVAL return will break
chvt.
> chvt uses the VT_ACTIVATE ioctl which calls set_console(), and it is
> valid for chvt to be used to change away from a graphics-mode console
--
> that shouldn't be an error condition.

Currently the VT_ACTIVATE ioctl will eventually result in a call to
change_console() which will ignore the change if the console is in
KD_GRAPHICS+VT_AUTO mode.  Thus, chvt should fail.  However, the return
code from set_console() is ignored in vt_ioctl so no error code will
result (which is the same as current behavior).

-- Andrew
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/