linux-kernel - Re: vm86 segfaults with NX bit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Fri, 21 Nov 2008 18:42:04 +0100
From:	Egbert Eich <eich@...e.de>
To:	Takashi Iwai <tiwai@...e.de>
Cc:	Arjan van de Ven <arjan@...radead.org>,
	linux-kernel@...r.kernel.org, Luc Verhaegen <lverhaegen@...e.de>,
	Egbert Eich <eich@...e.de>
Subject: Re: vm86 segfaults with NX bit

Takashi Iwai writes:
 > At Thu, 20 Nov 2008 08:58:44 -0800,
 > Arjan van de Ven wrote:
 > > 
 > > On Thu, 20 Nov 2008 16:39:06 +0100
 > > Takashi Iwai <tiwai@...e.de> wrote:
 > > 
 > > > Hi,
 > > > 
 > > > X guys reported that VESA driver segfaults in vm86 mode with a kernel
 > > > with CONFIG_X86_PAE, and it turned to be the NX bit.  See Novell
 > > > bugzilla #443440:
 > > > 	https://bugzilla.novell.com/show_bug.cgi?id=443440
 > > > 
 > > > I can confirm that the latest 2.6.28-rc still causes the same problem,
 > > > and it disappears when booted with noexec=off option.
 > > > 
 > > > Also, when NX bit is reset for the first 1MB (or smaller) in
 > > > do_sys_vm86(), it also works as expected.  But I have little clue
 > > > where to restore the bit again.
 > > > 
 > > > Any good suggestions / solutions?
 > > > 
 > > 
 > > did the code that mapped this memory setup use PROT_EXEC ?
 > 
 > A good question...
 > Luc, Egbert, how does VESA driver handle it?

Yes, we set the PROT_EXEC for the VBIOS. But the segfault doesn't happen
in VBIOS as far as I can tell. It happens on the 'exit instruction', a
hlt (0xf6) which is trapped by vm86 so it returns to the caller.
This is done by putting 0xf6 into address 0x600 and putting this address
on the stack.
The memory that's mapped at 0x600 in the Xserver's address space is
allocated thru shmget() and attached to the right place in the processes
memory space for vm86.
We should call mprotect() on these ranges to set this memory executable.
Hey, this code was orginally written almost 10 years ago!

The patch below fixes the issue.

Cheers,
	Egbert.

View attachment "diff.vm86_exec" of type "text/plain" (1415 bytes)