linux-kernel - math-emu issue with fp divide

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-Id: <9BECB10D-536E-4242-A2F5-6E1EF8CBDB40@kernel.crashing.org>
Date:	Wed, 4 Jun 2008 15:38:22 -0500
From:	Kumar Gala <galak@...nel.crashing.org>
To:	David Miller <davem@...emloft.net>
Cc:	lkml List <linux-kernel@...r.kernel.org>, libc-alpha@...rceware.org
Subject: math-emu issue with fp divide

David,

You seem to be the last person to touch the math-emu code in the  
kernel so I figured you might have some insights.  I'm trying to move  
the powerpc math-emu code to use the include/math-emu bits.

In doing so I've been using TestFloat to see how good or bad we are  
doing.  For the most part the current math-emu code that PPC uses has  
a number of issues that the code in include/math-emu seems to solve  
(plus bugs we've had for ever that no one every realized).

Anyways, I've come across a case that we are flagging underflow and  
inexact because we think we have a denormalized result from a double  
precision divide:

000.FFFFFFFFFFFFF / 3FE.FFFFFFFFFFFFE
	soft: 001.0000000000000 .....  syst: 001.0000000000000 ...ux

What it looks like is the results out of FP_DIV_D are:

D:
sign:	  0
mantissa: 01000000 00000000
exp:	 -1023 (0)

The problem seems like we aren't normalizing the result and bumping  
the exp.  So I was wondering if you had any idea if FP_DIV_D should be  
doing that or if when we call FP_PACK_DP/_FP_PACK_CANONICAL if that  
could should be doing it.

thanks

- k
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/