linux-kernel - Re: gcc-10: kernel stack is corrupted and fails to boot

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Wed, 13 May 2020 15:45:14 +0300
From:   Kalle Valo <kvalo@...eaurora.org>
To:     Arnd Bergmann <arnd@...db.de>
Cc:     linux-wireless <linux-wireless@...r.kernel.org>,
        "linux-kernel\@vger.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: gcc-10: kernel stack is corrupted and fails to boot

Arnd Bergmann <arnd@...db.de> writes:

> On Wed, May 13, 2020 at 8:50 AM Kalle Valo <kvalo@...eaurora.org> wrote:
>>
>> Kalle Valo <kvalo@...eaurora.org> writes:
>>
>> > This motivated me to switch to using GCC 10.x and I noticed that you had
>> > already upgraded crosstool so it was a trivial thing to do, awesome :)
>> >
>> > https://mirrors.edge.kernel.org/pub/tools/crosstool/
>>
>> And now I have a problem :) I first noticed that my x86 testbox is not
>> booting when I compile the kernel with GCC 10.1.0 from crosstool. I
>> didn't get any error messages so I just downgraded the compiler and the
>> kernel was booting fine again. Next I decided to try GCC 10.1 with my
>> x86 laptop and it also failed to boot, but this time I got kernel logs
>> and saw this:
>>
>> Kernel panic - not syncing: stack-protector: Kernel stack is
>> corrupted in: start_secodary+0x178/0x180
>>
>> Call Trace:
>> dump_stack
>> panic
>> ? _raw_spin_unlock_irqrestore
>> ? start_secondary
>> __stack_chk_fail
>> start_secondary
>> secondary_startup
>>
>> (I wrote the above messages manually from a picture so expect typos)
>>
>> Then also on my x86 laptop I downgraded the compiler to GCC 8.1.0 (from
>> crosstool), rebuilt the exactly same kernel version and the kernel
>> booted without issues.
>>
>> I'm using 5.7.0-rc4-wt-ath+ which is basically v5.7-rc4 plus latest
>> wireless patches, and I doubt the wireless patches are making any
>> difference this early in the boot. All compilers I use are prebuilt
>> binaries from kernel.org crosstool repo[1] with addition of ccache
>> v3.4.1 to speed up my builds.
>>
>> Any ideas? How should I debug this further?
>
> At least if it fails reproducibly, it's probably not too hard to drill
> down further. Some ideas:
>
> * I'd first try to reproduce it in qemu. Since you don't even need
>   any user space or modules, I would simply try
>   $ qemu-system-x86_64 -nographic -monitor none -append
> "console=ttyS0" -serial stdio -smp 4 -kernel arch/x86/boot/bzImage
>   I tried it here with an x86 defconfig linux-next kernel but did not
>   run into the problem you described.

Thanks, I'll try that but I expect it will take few days before I can do
it.

> If you share your .config, I can try reproducing with that as well.
> Once there is a reproducer in qemu, it should be trivial to step
> through it using gdb.

I have attached the .config I used with GCC 10.1. If you are able to
test it please do let me know how it went.

> * There are still two prerelease compiler versions on kernel.org,
>   from February and from April. You can try each one to see
>   if this was a recent regression. It's also possible that there is
>   a problem with my specific builds of gcc-10.1, and that the
>   compiler is actually fine for others.The gcc-10 packages in
>   Fedora/Debian/Ubuntu are probably better tested.

I'm still using Ubuntu 16.04 so not sure how easy it is to find a
package for that, but maybe this is a good reason to finally my upgrade
my laptop :)

-- 
https://wireless.wiki.kernel.org/en/developers/documentation/submittingpatches


View attachment ".config.old" of type "text/plain" (144248 bytes)