linux-kernel - floating point computation error caused by eagerfpu

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <CAB5AJuUr7qwp+WWLBVdZouJ+mnB6GWQu5AKjuWCY2N1dmsw30g@mail.gmail.com>
Date:   Fri, 23 Mar 2018 15:23:38 +0800
From:   Lei Chen <chenl.lei@...il.com>
To:     linux-kernel@...r.kernel.org
Subject: floating point computation error caused by eagerfpu

Hi,
I'm trying to figure out the root cause of a floating point
calculation error on kernel 4.4.98. My coworker runs a SHA1 test tool.
The generated sha1 does not match the expected value. Strangely, this
test just goes well on one VM. After a lot of comparison between this
VM and the bare metal x86-64 environment, we find the suspicious point
-- the VM uses 'lazy' mode FPU context switch while bare metal server
uses 'eager' mode. Then I rebuilt the kernel with "eagerfpu=DISABLE"
by default. I'm happily to see the test passes across different
platforms(different VMs and different x86 servers).

We don't have any custom FPU setting or modification to the native
Linux 4.4.98 kernel code. Per my understanding, during boot, system
will choose eagerfpu mode automatically according  to the CPU's
capability. It should have just worked well if the CPU supports eager
mode. But the test result shows that there might be FPU context
corruption. Having googled around, I don't find similar report. Could
FPU experts shed some light on this issue?

Thanks,
Lei Chen