linux-kernel - Re: PROBLEM: Caught SIGFPE exceptions aren't reset

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <e21d91280708251031k5e3bd962o6a268e6506696a8c@mail.gmail.com>
Date:	Sat, 25 Aug 2007 13:31:43 -0400
From:	"Clark Cooper" <clark.cooper@...il.com>
To:	linux-kernel@...r.kernel.org
Subject: Re: PROBLEM: Caught SIGFPE exceptions aren't reset

Thank you for your prompt answer. You wrote:

> Your handler can do this by masking
>  exceptions, changing the operands of the failed FP instruction,
> or by changing the PC so that the failed instruction is skipped
> (your handler may want to emulate the instruction in this case).

Given the current SIGFPE handling in the kernel, a userland handler
CAN'T mask the exceptions or clear the exception flags. This is
demonstrated in my example program where I call both feclearexception
and fedisableexception in the handler. The reason these don't work is
that any change you make to these FPU registers is overwritten with
the saved FPU context on return from the signal handler. Changing the
PC won't help since the exception flag is still set and unmasked on
signal return.

Now either the kernel should clear the corresponding exception flags
or it should present the correct FPU state to the handler for
manipulation. Currently neither of these appear to be the case. So if
there is no problem with the kernel, then the signal man pages should
indicate that while you can catch a SIGFPE, you can't actually return
from it.

If I am completely wrong on this, could you please indicate what kind
of change could be made to the handler in my example program to allow
it to reach the final printf statement in the main program?

Thank you,
Clark Cooper
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/