linux-kernel - Re: transparent huge pages breaks KVM on AMD.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20160513083545.GA1307@gallifrey>
Date:	Fri, 13 May 2016 09:35:45 +0100
From:	"Dr. David Alan Gilbert" <dave@...blig.org>
To:	Marc Haber <mh+linux-kernel@...schlus.de>
Cc:	Borislav Petkov <bp@...en8.de>,
	Paolo Bonzini <pbonzini@...hat.com>,
	linux-kernel@...r.kernel.org, kvm ML <kvm@...r.kernel.org>
Subject: Re: transparent huge pages breaks KVM on AMD.

* Marc Haber (mh+linux-kernel@...schlus.de) wrote:
> Hi David,
> 
> On Sat, Apr 23, 2016 at 07:52:46PM +0100, Dr. David Alan Gilbert wrote:
> > Hmm, your problem does sound like bad hardware, but....
> > If you've got a nice reliable crash, can you try turning transparent huge pages
> > off on the host;
> >    echo never > /sys/kernel/mm/transparent_hugepage/enabled
> 
> I must have missed this hint in the middle of the "your hardware is
> bad" avalance that came over me.
> 
> I spent two weeks bisecting "good" kernels since during the repeated
> reconfigurations, transparent huge pages got turned off in kernel
> configuration. After running each kernel for 24 hours, I eventually
> ended up with a working 4.5 kernel. The configuration diff was short,
> showing transparent huge pages, and - finally - upon re-reading the
> thread I found your hint.

OK, good.  When I sent that mail I'd hit a THP bug but in a
corner of migration and at the time we didn't know why and there was
no reason to think it would cause any other symptoms, but since it was
also between 4.4 and 4.5 it did seem worth mentioning as a long shot,
but it was no more than a long shot.

> I have now the result that 4.5, 4.5.1 and 4.5.4 corrupt KVM guest
> memory reliably in the first hour of running under disk load, causing
> the VM to either drop dead in the water, or to read randomness from
> disk. Rebooting fixes the VM. This happens as soon as transparent huge
> pages are turned on in the host.
> 
> Turning off transparent huge pages by echo never >
> /sys/kernel/mm/transparent_hugepage/enabled fixes the issue even
> without rebooting the host. Start up the VM again and it works just
> fine.
> 
> Is this an issue in (a) transparent huge pages, (b) KVM or (c) qemu?
> Where should this issue be forwarded? Or do we just accept it and turn
> transparent huge pages off?

Try Andrea's fix for (a).

Dave

> 
> Greetings
> Marc
> 
> -- 
> -----------------------------------------------------------------------------
> Marc Haber         | "I don't trust Computers. They | Mailadresse im Header
> Leimen, Germany    |  lose things."    Winona Ryder | Fon: *49 6224 1600402
> Nordisch by Nature |  How to make an American Quilt | Fax: *49 6224 1600421
-- 
 -----Open up your eyes, open up your mind, open up your code -------   
/ Dr. David Alan Gilbert    |       Running GNU/Linux       | Happy  \ 
\        dave @ treblig.org |                               | In Hex /
 \ _________________________|_____ http://www.treblig.org   |_______/