linux-kernel - Emulating unsupported x86 opcodes in software

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 25 Jan 2012 10:58:32 +0100
From:	Stefan Seidel <stefans@...roid-x86.org>
To:	linux-kernel@...r.kernel.org
Subject: Emulating unsupported x86 opcodes in software

Hello,

I'm looking for advice regarding the emulation of CPU instructions
which the current processor doesn't support. Specifically, I want to
emulate SSE3 instructions on non-SSE3-capable CPUs. The OSx86
project[1] has developed an SSE3 emulation based on SSE2 instructions.

The general idea is:
 - install a "stage 1" trap handler for INT 6h (invalid opcode)
 - "stage 1" handler returns the address of the "stage 2" handler, the
actual emulation routine
 - "stage 2" handler runs in user-mode, reads the invalid (SSE3)
instruction and operands and emulates it
 - "stage 2" handler stores the result in the correct destination
(register or memory) and restores all dirty registers to their
original state
More in-depth description is available as PDF[2].

Now it is not a problem to replace the INT 6h handler, but these are
my open questions:
1. How can I have the stage 2 handler run in user-mode (ring 3)
2. How do I allocate/obtain the address of the code segment containing
the stage 2 handler OR
3. Is it even possible to for code which is implemented in the kernel
to be called and executed in user-mode, and if not, what is the
alternative.

Both stage 1 and 2 handlers are available as binary (currently, I'm
trying to contact the original author for the source code).

Thank you very much for any help.

Stefan Seidel
P.S.: CC'ing me in your answer would be appreciated.

[1] http://code.google.com/p/xnu-dev/
[2] http://code.google.com/p/xnu-dev/downloads/detail?name=SSE3%20Emulator.pdf
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/