linux-kernel - Re: problem booting 5.10

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAHk-=wi=R7uAoaVK9ewDPdCYDn1i3i19uoOzXEW5Nn8UV-1_AA@mail.gmail.com>
Date:   Tue, 8 Dec 2020 10:31:49 -0800
From:   Linus Torvalds <torvalds@...ux-foundation.org>
To:     Julia Lawall <julia.lawall@...ia.fr>,
        "Martin K. Petersen" <martin.petersen@...cle.com>,
        "James E.J. Bottomley" <jejb@...ux.ibm.com>
Cc:     Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        nicolas.palix@...v-grenoble-alpes.fr,
        linux-scsi <linux-scsi@...r.kernel.org>
Subject: Re: problem booting 5.10

On Tue, Dec 8, 2020 at 9:37 AM Julia Lawall <julia.lawall@...ia.fr> wrote:
>
> We have not succeeded to boot 5.10 on our Intel(R) Xeon(R) CPU E7-8870 v4 @
> 2.10GHz server.  Previous versions (eg 4.19 - 5.9) boot fine.  We have
> tried various rcs.

So the problem started with rc1?

Could you try bisecting - even partially? If you do only six
bisections, the number of suspect commits drops from 15k to about 230
- which likely pinpoints the suspect area.

That said, your traces certainly makes me go "Hmm. Some thing broke in
SCSI device scanning", with the primary one being the
wait_for_completion() one - the rest of the stuck processes seem to be
stuck in async_synchronize_cookie_domain() and are presumably waiting
for this kthread that is waiting for the scan to finish.

So I'm adding SCSI people to the cc, just in case they go "Hmm..".

Martin & co - in the next email Julia also quotes

> [   51.355655][    T7] scsi 0:0:14:0: Direct-Access     ATA      ST2000LM015-2E81 SDM1 PQ: 0 ANSI: 6
> Gave up waiting for root file system device.  Common problems:[..]

which seems to be more of the same pattern with the SCSI scanning failure.

Of course, it could be some non-scsi patch that causes this, but.. A
bisect would hopefully clarify.

Leaving the (simplified) backtrace quoted below.

                   Linus

>The backtrace for rc7 is shown below.
>
> [  253.207171][  T979] INFO: task kworker/u321:2:1278 blocked for more than 120 seconds.
> [  253.224089][  T979]       Tainted: G            E     5.10.0-rc7 #3
> [  253.239209][  T979] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
> [  253.256990][  T979] task:kworker/u321:2  state:D stack:    0 pid: 1278 ppid:     2 flags:0x00004000
> [  253.275552][  T979] Workqueue: events_unbound async_run_entry_fn
> [  253.290687][  T979] Call Trace:
> [  253.302491][  T979]  __schedule+0x31e/0x890
> [  253.315353][  T979]  schedule+0x3c/0xa0
> [  253.327688][  T979]  schedule_timeout+0x274/0x310
> [  253.379283][  T979]  wait_for_completion+0x8a/0xf0
> [  253.392327][  T979]  scsi_complete_async_scans+0x107/0x170
> [  253.406115][  T979]  __scsi_add_device+0xf7/0x130
> [  253.418974][  T979]  ata_scsi_scan_host+0x98/0x1c0
> [  253.431948][  T979]  async_run_entry_fn+0x39/0x160
> [  253.444853][  T979]  process_one_work+0x24c/0x490