linux-kernel - Re: 2.6.22 x86_64 : kernel initial decompression hangs on vmware

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <46B4A7E4.4030907@vmware.com>
Date:	Sat, 04 Aug 2007 09:23:00 -0700
From:	Zachary Amsden <zach@...are.com>
To:	Gabriel Barazer <gabriel@...va.fr>, Andi Kleen <ak@...e.de>,
	Jim Mattson <jmattson@...are.com>
CC:	linux-kernel@...r.kernel.org,
	Virtualization Mailing List <virtualization@...ts.osdl.org>,
	Avi Kivity <avi@...ranet.com>,
	kvm-devel <kvm-devel@...ts.sourceforge.net>
Subject: Re: 2.6.22 x86_64 : kernel initial decompression hangs on vmware

Gabriel Barazer wrote:
> Hi,
>
> After upgrading kernel to 2.6.22 on a Vmware workstation guest version 
> 5.5 and 6 , the kernel decompression stage ("Decompressing Linux...") 
> is hanging for a very long time (~5 minutes) before finally  
> succeeding (displaying "done.\nBooting the kernel.\n"). During this 
> time, the VM process is eating all the CPU time during the 
> decompression, like an infinite loop.
> Between these 2 strings is the gunzip() function at 
> boot/compressed/misc.c which does the real job, and the problem seemed 
> to appear since commit 1ab60e0f72f71ec54831e525a3e1154f1c092408. 
> (2.6.22-rc1 hangs, 2.6.21.6 works). The problem occurs with or without 
> CONFIG_RELOCATABLE enabled.
>
> What are the possible solutions to confirm where the problem is coming 
> from ?

Since I was just involved in the boot decompressor for another bug, I 
took a look at this.  2.6.22 switches it to be 64-bit code.  VT is very 
picky about what state it can run in.  Not using VT on Intel 64-bit 
hardware cripples performance, running at far below normal speed, and 
taking minutes to decompress the kernel, which is nearly instantaneous 
otherwise.

To get back into VT in this case, not only do we need to load FS and GS, 
we also need to setup an initial LDT and task.  Can you try the attached 
patch and see that it does the right thing?

I've also cc'd the KVM developers, as the same problem will affect them, 
and hopefully the same patch will fix it.

Thanks, Zach

Zach

View attachment "boot-decompress-vt-fix.patch" of type "text/x-patch" (1178 bytes)