linux-kernel - Re: 💥 PANICKED: Waiting for review: Test report for kernel 5.9.11 (stable-queue)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [day] [month] [year] [list]
Message-ID: <498412291.60269669.1606793960061.JavaMail.zimbra@redhat.com>
Date:   Mon, 30 Nov 2020 22:39:20 -0500 (EST)
From:   Xiumei Mu <xmu@...hat.com>
To:     CKI Project <cki-project@...hat.com>, linux-kernel@...r.kernel.org,
        linux-mm@...ck.org
Cc:     skt-results-master@...hat.com, Rachel Sibley <rasibley@...hat.com>,
        Jianwen Ji <jiji@...hat.com>, Hangbin Liu <haliu@...hat.com>,
        David Arcari <darcari@...hat.com>, stable@...r.kernel.org,
        akpm@...ux-foundation.org, gregkh@...uxfoundation.org
Subject: Re: 💥 PANICKED: Waiting for review: Test report for kernel 5.9.11 (stable-queue)


----- Original Message -----
> From: "CKI Project" <cki-project@...hat.com>
> To: skt-results-master@...hat.com
> Cc: "Yi Zhang" <yi.zhang@...hat.com>, "Xiong Zhou" <xzhou@...hat.com>, "Rachel Sibley" <rasibley@...hat.com>, "Xiumei
> Mu" <xmu@...hat.com>, "Jianwen Ji" <jiji@...hat.com>, "Hangbin Liu" <haliu@...hat.com>, "David Arcari"
> <darcari@...hat.com>
> Sent: Sunday, November 29, 2020 11:32:43 AM
> Subject: 💥 PANICKED: Waiting for review: Test report for kernel 5.9.11 (stable-queue)
> 
> 
> ┌───────────────────────────────────────────────────────────────┐
> │ REVIEW REQUIRED FOR FAILED TEST                               │
> ├───────────────────────────────────────────────────────────────┤
> │ This failed kernel test has been held for review by kernel    │
> │ test maintainers and the CKI team. Please investigate using   │
> │ the pipeline link below this box.                             │
> │                                                               │
> │ If the test failure is related to a non-kernel bug, no action │
> │ is needed. If a kernel bug is found, please reply all with    │
> │ your assessment and we will release the report.               │
> │ For more details: https://docs.engineering.redhat.com/x/eG5qB │
> └───────────────────────────────────────────────────────────────┘
> 
> Pipeline:
> https://xci32.lab.eng.rdu2.redhat.com/cki-project/cki-pipeline/-/pipelines/618811
> 
> Check out if the issue is autotriaged in the dashboard:
>     https://datawarehouse.internal.cki-project.org/kcidb/revisions/7444
> 
> Hello,
> 
> We ran automated tests on a recent commit from this kernel tree:
> 
>        Kernel repo:
>        https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git
>             Commit: 6d1358afba7c - drm/amd/display: Avoid HDCP initialization
>             in devices without output
> 
> The results of these automated tests are provided below.
> 
>     Overall result: FAILED (see details below)
>              Merge: OK
>            Compile: OK
>              Tests: PANICKED
> 
> All kernel binaries, config files, and logs are available for download here:
> 
>   https://arr-cki-prod-datawarehouse-public.s3.amazonaws.com/index.html?prefix=datawarehouse-public/2020/11/28/618811
> 
> One or more kernel tests failed:
> 
>     x86_64:
>      💥 Networking socket: fuzz

The test case is socket fuzz test, no reproducer, and no vmcore.

The Call Trace is here:
[15358.193947] can: controller area network core (rev 20170425 abi 9) 
[15358.195259] NET: Registered protocol family 29 
[15358.852499] list_del corruption. next->prev should be ffff9a927074d550, but was ffff9a922773e360 
[15358.853108] ------------[ cut here ]------------ 
[15358.853108] kernel BUG at lib/list_debug.c:54! 
[15358.853108] invalid opcode: 0000 [#1] SMP PTI 
[15358.853108] CPU: 1 PID: 252559 Comm: kworker/1:3 Kdump: loaded Tainted: G          IOE     5.9.11 #1 
[15358.853108] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15358.853108] Workqueue: cgwb_release cgwb_release_workfn 
[15358.853108] RIP: 0010:__list_del_entry_valid.cold+0x1d/0x47 
[15358.853108] Code: c7 c7 30 97 3e 85 e8 3b 4c fe ff 0f 0b 48 89 fe 48 c7 c7 c0 97 3e 85 e8 2a 4c fe ff 0f 0b 48 c7 c7 70 98 3e 85 e8 1c 4c fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 30 98 3e 85 e8 08 4c fe ff 0f 0b 
[15358.853108] RSP: 0018:ffffbbf101347e40 EFLAGS: 00010092 
[15358.853108] RAX: 0000000000000054 RBX: ffff9a927074d540 RCX: 0000000000000000 
[15358.853108] RDX: ffff9a927fc67060 RSI: ffff9a927fc58d00 RDI: ffff9a927fc58d00 
[15358.853108] RBP: 0000000000000206 R08: 000000000000468c R09: 0000000000000b12 
[15358.853108] R10: 0000000000000000 R11: ffffbbf101347cdd R12: 00000000ffffffff 
[15358.853108] R13: ffff9a927074d400 R14: ffff9a9244067000 R15: ffff9a927074d6a0 
[15358.853108] FS:  0000000000000000(0000) GS:ffff9a927fc40000(0000) knlGS:0000000000000000 
[15358.853108] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15358.853108] CR2: 00007f74a76990f8 CR3: 0000000020be4000 CR4: 00000000000406e0 
[15358.853108] Call Trace: 
[15358.853108]  percpu_counter_destroy+0x24/0x80 
[15358.853108]  wb_exit+0x4d/0x90 
[15358.853108]  cgwb_release_workfn+0xa1/0x140 
[15358.853108]  process_one_work+0x1b4/0x370 
[15358.853108]  worker_thread+0x53/0x3e0 
[15358.853108]  ? process_one_work+0x370/0x370 
[15358.853108]  kthread+0x11b/0x140 
[15358.853108]  ? __kthread_bind_mask+0x60/0x60 
[15358.853108]  ret_from_fork+0x22/0x30 
[15358.853108] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15358.853108] ---[ end trace e621d981e1857d7e ]--- 
[15358.853108] RIP: 0010:__list_del_entry_valid.cold+0x1d/0x47 
[15358.853108] Code: c7 c7 30 97 3e 85 e8 3b 4c fe ff 0f 0b 48 89 fe 48 c7 c7 c0 97 3e 85 e8 2a 4c fe ff 0f 0b 48 c7 c7 70 98 3e 85 e8 1c 4c fe ff <0f> 0b 48 89 f2 48 89 fe 48 c7 c7 30 98 3e 85 e8 08 4c fe ff 0f 0b 
[15358.853108] RSP: 0018:ffffbbf101347e40 EFLAGS: 00010092 
[15358.853108] RAX: 0000000000000054 RBX: ffff9a927074d540 RCX: 0000000000000000 
[15358.853108] RDX: ffff9a927fc67060 RSI: ffff9a927fc58d00 RDI: ffff9a927fc58d00 
[15358.853108] RBP: 0000000000000206 R08: 000000000000468c R09: 0000000000000b12 
[15358.853108] R10: 0000000000000000 R11: ffffbbf101347cdd R12: 00000000ffffffff 
[15358.853108] R13: ffff9a927074d400 R14: ffff9a9244067000 R15: ffff9a927074d6a0 
[15358.853108] FS:  0000000000000000(0000) GS:ffff9a927fc40000(0000) knlGS:0000000000000000 
[15358.853108] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15358.853108] CR2: 00007f74a76990f8 CR3: 0000000020be4000 CR4: 00000000000406e0 
[15359.150252] sock: process `socket' is using obsolete setsockopt SO_BSDCOMPAT 
[15378.513070] NMI watchdog: Watchdog detected hard LOCKUP on cpu 3 
[15378.513070] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15378.513094] CPU: 3 PID: 0 Comm: swapper/3 Kdump: loaded Tainted: G      D   IOE     5.9.11 #1 
[15378.513095] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15378.513095] RIP: 0010:cpu_idle_poll.isra.0+0x36/0xf0 
[15378.513096] Code: 48 7b 66 66 66 66 90 e8 18 25 5c ff fb 66 66 90 66 66 90 65 48 8b 1c 25 c0 7b 01 00 48 8b 03 a8 08 74 0b eb 1c f3 90 48 8b 03 <a8> 08 75 13 8b 05 08 cb cd 00 85 c0 75 ed e8 e7 23 5e ff 85 c0 75 
[15378.513097] RSP: 0018:ffffbbf100087ef8 EFLAGS: 00000202 
[15378.513098] RAX: 0000000000204000 RBX: ffff9a927f720000 RCX: 00000dfaa484f7cd 
[15378.513098] RDX: ffff9a927f46e070 RSI: 00000dfaa484f7cd RDI: 0000000000000000 
[15378.513099] RBP: 0000000000000003 R08: 00000dfaa484fc38 R09: 00000dfaa484f2ed 
[15378.513099] R10: 0000000000000005 R11: 000000000001f865 R12: 0000000000000000 
[15378.513100] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 
[15378.513101] FS:  0000000000000000(0000) GS:ffff9a927fcc0000(0000) knlGS:0000000000000000 
[15378.513101] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15378.513102] CR2: 00007f0f00017198 CR3: 000000006d8cc000 CR4: 00000000000406e0 
[15378.513102] DR0: 0000000000420810 DR1: 0000000000000000 DR2: 0000000000000000 
[15378.513103] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 
[15378.513103] Call Trace: 
[15378.513103]  do_idle+0x44/0x270 
[15378.513104]  cpu_startup_entry+0x19/0x20 
[15378.513104]  secondary_startup_64+0xb6/0xc0 
[15384.392107] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [modprobe:439666] 
[15384.392107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15384.392107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOE     5.9.11 #1 
[15384.392107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15384.392107] RIP: 0010:smp_call_function_many_cond+0x289/0x2d0 
[15384.392107] Code: e8 7c 07 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 f3 90 <8b> 42 08 a8 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 89 f7 
[15384.392107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15384.392107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15384.392107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15384.392107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15384.392107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15384.392107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15384.392107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15384.392107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15384.392107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15384.392107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15384.392107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 
[15384.392107] Call Trace: 
[15384.392107]  ? invalidate_user_asid+0x40/0x40 
[15384.392107]  ? invalidate_user_asid+0x40/0x40 
[15384.392107]  on_each_cpu+0x2b/0x60 
[15384.392107]  __purge_vmap_area_lazy+0x5d/0x690 
[15384.392107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15384.392107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15384.392107]  change_page_attr_set_clr+0x9e/0x190 
[15384.392107]  set_memory_ro+0x26/0x30 
[15384.392107]  module_enable_ro.part.0+0x5f/0xb0 
[15384.392107]  load_module+0x20a7/0x2550 
[15384.392107]  ? ima_post_read_file+0xca/0xe0 
[15384.392107]  __do_sys_finit_module+0x9c/0xe0 
[15384.392107]  do_syscall_64+0x33/0x40 
[15384.392107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15384.392107] RIP: 0033:0x7f267dc3130d 
[15384.392107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15384.392107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15384.392107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15384.392107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15384.392107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15384.392107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15384.392107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[15412.392107] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [modprobe:439666] 
[15412.392107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15412.392107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15412.392107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15412.392107] RIP: 0010:smp_call_function_many_cond+0x289/0x2d0 
[15412.392107] Code: e8 7c 07 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 f3 90 <8b> 42 08 a8 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 89 f7 
[15412.392107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15412.392107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15412.392107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15412.392107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15412.392107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15412.392107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15412.392107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15412.392107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15412.392107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15412.392107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15412.392107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 
[15412.392107] Call Trace: 
[15412.392107]  ? invalidate_user_asid+0x40/0x40 
[15412.392107]  ? invalidate_user_asid+0x40/0x40 
[15412.392107]  on_each_cpu+0x2b/0x60 
[15412.392107]  __purge_vmap_area_lazy+0x5d/0x690 
[15412.392107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15412.392107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15412.392107]  change_page_attr_set_clr+0x9e/0x190 
[15412.392107]  set_memory_ro+0x26/0x30 
[15412.392107]  module_enable_ro.part.0+0x5f/0xb0 
[15412.392107]  load_module+0x20a7/0x2550 
[15412.392107]  ? ima_post_read_file+0xca/0xe0 
[15412.392107]  __do_sys_finit_module+0x9c/0xe0 
[15412.392107]  do_syscall_64+0x33/0x40 
[15412.392107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15412.392107] RIP: 0033:0x7f267dc3130d 
[15412.392107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15412.392107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15412.392107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15412.392107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15412.392107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15412.392107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15412.392107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[15440.392107] watchdog: BUG: soft lockup - CPU#2 stuck for 22s! [modprobe:439666] 
[15440.392107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15440.392107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15440.392107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15440.392107] RIP: 0010:smp_call_function_many_cond+0x287/0x2d0 
[15440.392107] Code: 76 08 e8 7c 07 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 <f3> 90 8b 42 08 a8 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 
[15440.392107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15440.392107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15440.392107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15440.392107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15440.392107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15440.392107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15440.392107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15440.392107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15440.392107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15440.392107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15440.392107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 
[15440.392107] Call Trace: 
[15440.392107]  ? invalidate_user_asid+0x40/0x40 
[15440.392107]  ? invalidate_user_asid+0x40/0x40 
[15440.392107]  on_each_cpu+0x2b/0x60 
[15440.392107]  __purge_vmap_area_lazy+0x5d/0x690 
[15440.392107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15440.392107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15440.392107]  change_page_attr_set_clr+0x9e/0x190 
[15440.392107]  set_memory_ro+0x26/0x30 
[15440.392107]  module_enable_ro.part.0+0x5f/0xb0 
[15440.392107]  load_module+0x20a7/0x2550 
[15440.392107]  ? ima_post_read_file+0xca/0xe0 
[15440.392107]  __do_sys_finit_module+0x9c/0xe0 
[15440.392107]  do_syscall_64+0x33/0x40 
[15440.392107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15440.392107] RIP: 0033:0x7f267dc3130d 
[15440.392107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15440.392107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15440.392107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15440.392107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15440.392107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15440.392107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15440.392107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[15468.392107] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [modprobe:439666] 
[15468.392107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15468.392107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15468.392107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15468.392107] RIP: 0010:smp_call_function_many_cond+0x28c/0x2d0 
[15468.392107] Code: 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 f3 90 8b 42 08 <a8> 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 89 f7 e8 0c 08 
[15468.392107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15468.392107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15468.392107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15468.392107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15468.392107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15468.392107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15468.392107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15468.392107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15468.392107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15468.392107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15468.392107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 
[15468.392107] Call Trace: 
[15468.392107]  ? invalidate_user_asid+0x40/0x40 
[15468.392107]  ? invalidate_user_asid+0x40/0x40 
[15468.392107]  on_each_cpu+0x2b/0x60 
[15468.392107]  __purge_vmap_area_lazy+0x5d/0x690 
[15468.392107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15468.392107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15468.392107]  change_page_attr_set_clr+0x9e/0x190 
[15468.392107]  set_memory_ro+0x26/0x30 
[15468.392107]  module_enable_ro.part.0+0x5f/0xb0 
[15468.392107]  load_module+0x20a7/0x2550 
[15468.392107]  ? ima_post_read_file+0xca/0xe0 
[15468.392107]  __do_sys_finit_module+0x9c/0xe0 
[15468.392107]  do_syscall_64+0x33/0x40 
[15468.392107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15468.392107] RIP: 0033:0x7f267dc3130d 
[15468.392107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15468.392107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15468.392107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15468.392107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15468.392107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15468.392107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15468.392107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[15496.392107] watchdog: BUG: soft lockup - CPU#2 stuck for 23s! [modprobe:439666] 
[15496.392107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15496.392107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15496.392107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15496.392107] RIP: 0010:smp_call_function_many_cond+0x28c/0x2d0 
[15496.392107] Code: 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 f3 90 8b 42 08 <a8> 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 89 f7 e8 0c 08 
[15496.392107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15496.392107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15496.392107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15496.392107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15496.392107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15496.392107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15496.392107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15496.392107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15496.392107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15496.392107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15496.392107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000600 
[15496.392107] Call Trace: 
[15496.392107]  ? invalidate_user_asid+0x40/0x40 
[15496.392107]  ? invalidate_user_asid+0x40/0x40 
[15496.392107]  on_each_cpu+0x2b/0x60 
[15496.392107]  __purge_vmap_area_lazy+0x5d/0x690 
[15496.392107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15496.392107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15496.392107]  change_page_attr_set_clr+0x9e/0x190 
[15496.392107]  set_memory_ro+0x26/0x30 
[15496.392107]  module_enable_ro.part.0+0x5f/0xb0 
[15496.392107]  load_module+0x20a7/0x2550 
[15496.392107]  ? ima_post_read_file+0xca/0xe0 
[15496.392107]  __do_sys_finit_module+0x9c/0xe0 
[15496.392107]  do_syscall_64+0x33/0x40 
[15496.392107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15496.392107] RIP: 0033:0x7f267dc3130d 
[15496.392107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15496.392107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15496.392107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15496.392107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15496.392107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15496.392107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15496.392107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[-- MARK -- Sat Nov 28 20:20:00 2020] 
[15509.735110] NMI watchdog: Watchdog detected hard LOCKUP on cpu 1 
[15509.735110] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15509.735110] CPU: 1 PID: 0 Comm: swapper/1 Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15509.735110] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15509.735110] RIP: 0010:cpu_idle_poll.isra.0+0x36/0xf0 
[15509.735110] Code: 48 7b 66 66 66 66 90 e8 18 25 5c ff fb 66 66 90 66 66 90 65 48 8b 1c 25 c0 7b 01 00 48 8b 03 a8 08 74 0b eb 1c f3 90 48 8b 03 <a8> 08 75 13 8b 05 08 cb cd 00 85 c0 75 ed e8 e7 23 5e ff 85 c0 75 
[15509.735110] RSP: 0018:ffffbbf100077ef8 EFLAGS: 00000202 
[15509.735110] RAX: 0000000000204000 RBX: ffff9a927f724b80 RCX: 00000e1b069062c3 
[15509.735110] RDX: ffff9a927f46e070 RSI: 00000e1b069062c3 RDI: 0000000000000000 
[15509.735110] RBP: 0000000000000001 R08: 00000e1b069066a3 R09: 00000e1b06905ce9 
[15509.735110] R10: 0000000000000005 R11: 000000000001f865 R12: 0000000000000000 
[15509.735110] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000 
[15509.735110] FS:  0000000000000000(0000) GS:ffff9a927fc40000(0000) knlGS:0000000000000000 
[15509.735110] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15509.735110] CR2: 0000000001543ec8 CR3: 0000000020be4000 CR4: 00000000000406e0 
[15509.735110] Call Trace: 
[15509.735110]  do_idle+0x44/0x270 
[15509.735110]  cpu_startup_entry+0x19/0x20 
[15509.735110]  secondary_startup_64+0xb6/0xc0 
[15509.736107] NMI watchdog: Watchdog detected hard LOCKUP on cpu 2 
[15509.736107] Modules linked in: can mlx4_ib ib_uverbs ib_core mlx4_en mlx4_core nls_utf8 isofs snd_seq_dummy minix binfmt_misc nfsv3 nfs_acl rds tun brd overlay fuse exfat vfat fat loop n_gsm pps_ldisc ppp_synctty mkiss ax25 ppp_async ppp_generic slcan slip slhc snd_hrtimer snd_seq snd_seq_device sctp snd_timer snd soundcore authenc pcrypt crypto_user sha3_generic salsa20_generic n_hdlc rpcsec_gss_krb5 auth_rpcgss nfsv4 dns_resolver nfs lockd grace nfs_ssc fscache rfkill sunrpc iTCO_wdt intel_pmc_bxt gpio_ich coretemp iTCO_vendor_support kvm_intel kvm irqbypass ipmi_ssif pcspkr hpilo bnx2 acpi_ipmi hpwdt lpc_ich ipmi_si ipmi_devintf ipmi_msghandler i5000_edac i5k_amb zram ip_tables xfs radeon i2c_algo_bit drm_kms_helper cec ttm serio_raw drm ata_generic pata_acpi hpsa scsi_transport_sas [last unloaded: dummy] 
[15509.736107] CPU: 2 PID: 439666 Comm: modprobe Kdump: loaded Tainted: G      D   IOEL    5.9.11 #1 
[15509.736107] Hardware name: HP ProLiant DL360 G5, BIOS P58 07/10/2009 
[15509.736107] RIP: 0010:smp_call_function_many_cond+0x289/0x2d0 
[15509.736107] Code: e8 7c 07 4c 00 3b 05 ca 2f 6f 01 89 c7 0f 83 0b fe ff ff 48 63 c7 49 8b 16 48 03 14 c5 00 59 47 85 8b 42 08 a8 01 74 09 f3 90 <8b> 42 08 a8 01 75 f7 eb c9 48 c7 c2 a0 8e 86 85 4c 89 fe 44 89 f7 
[15509.736107] RSP: 0018:ffffbbf100dc7ba0 EFLAGS: 00000202 
[15509.736107] RAX: 0000000000000011 RBX: 00000000000313c0 RCX: 0000000000000000 
[15509.736107] RDX: ffff9a927fc313c0 RSI: 0000000000000000 RDI: 0000000000000000 
[15509.736107] RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000 
[15509.736107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 
[15509.736107] R13: 0000000000000003 R14: ffff9a927fcab400 R15: 0000000000000008 
[15509.736107] FS:  00007f267db07740(0000) GS:ffff9a927fc80000(0000) knlGS:0000000000000000 
[15509.736107] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033 
[15509.736107] CR2: 00007f267dc9c220 CR3: 0000000037d0a000 CR4: 00000000000406e0 
[15509.736107] DR0: 0000000000000001 DR1: 0000000000000000 DR2: 0000000000000000 
[15509.736107] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 
[15509.736107] Call Trace: 
[15509.736107]  ? invalidate_user_asid+0x40/0x40 
[15509.736107]  ? invalidate_user_asid+0x40/0x40 
[15509.736107]  on_each_cpu+0x2b/0x60 
[15509.736107]  __purge_vmap_area_lazy+0x5d/0x690 
[15509.736107]  ? find_exported_symbol_in_section+0x45/0xd0 
[15509.736107]  _vm_unmap_aliases.part.0+0x104/0x140 
[15509.736107]  change_page_attr_set_clr+0x9e/0x190 
[15509.736107]  set_memory_ro+0x26/0x30 
[15509.736107]  module_enable_ro.part.0+0x5f/0xb0 
[15509.736107]  load_module+0x20a7/0x2550 
[15509.736107]  ? ima_post_read_file+0xca/0xe0 
[15509.736107]  __do_sys_finit_module+0x9c/0xe0 
[15509.736107]  do_syscall_64+0x33/0x40 
[15509.736107]  entry_SYSCALL_64_after_hwframe+0x44/0xa9 
[15509.736107] RIP: 0033:0x7f267dc3130d 
[15509.736107] Code: 00 c3 66 2e 0f 1f 84 00 00 00 00 00 90 f3 0f 1e fa 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 3b 7b 0c 00 f7 d8 64 89 01 48 
[15509.736107] RSP: 002b:00007ffeb0520438 EFLAGS: 00000246 ORIG_RAX: 0000000000000139 
[15509.736107] RAX: ffffffffffffffda RBX: 000055ac2ca54ed0 RCX: 00007f267dc3130d 
[15509.736107] RDX: 0000000000000000 RSI: 000055ac2b6f96a6 RDI: 0000000000000000 
[15509.736107] RBP: 0000000000040000 R08: 0000000000000000 R09: 0000000000000000 
[15509.736107] R10: 0000000000000000 R11: 0000000000000246 R12: 000055ac2b6f96a6 
[15509.736107] R13: 000055ac2ca54ce0 R14: 000055ac2ca54ed0 R15: 000055ac2ca55040 
[-- MARK -- Sat Nov 28 20:25:00 2020] 



> 
> We hope that these logs can help you find the problem quickly. For the full
> detail on our testing procedures, please scroll to the bottom of this
> message.
> 
> Please reply to this email if you have any questions about the tests that we
> ran or if you have any suggestions on how to make future tests more
> effective.
> 
>         ,-.   ,-.
>        ( C ) ( K )  Continuous
>         `-',-.`-'   Kernel
>           ( I )     Integration
>            `-'
> ______________________________________________________________________________
> 
> Compile testing
> ---------------
> 
> We compiled the kernel for 4 architectures:
> 
>     aarch64:
>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
>     ppc64le:
>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
>     s390x:
>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
>     x86_64:
>       make options: make -j30 INSTALL_MOD_STRIP=1 targz-pkg
> 
> 
> 
> Hardware testing
> ----------------
> We booted each kernel and ran the following tests:
> 
>   aarch64:
>     Host 1: https://beaker.engineering.redhat.com/recipes/9115542
> 
>        ⚡ Internal infrastructure issues prevented one or more tests (marked
>        with ⚡⚡⚡) from running on this architecture.
>        This is not the fault of the kernel that was tested.
> 
>        ⚡⚡⚡ Boot test
>        ⚡⚡⚡ selinux-policy: serge-testsuite
>        ⚡⚡⚡ storage: software RAID testing
>        ⚡⚡⚡ stress: stress-ng
>        🚧 ⚡⚡⚡ xfstests - ext4
>        🚧 ⚡⚡⚡ xfstests - xfs
>        🚧 ⚡⚡⚡ xfstests - btrfs
>        🚧 ⚡⚡⚡ IPMI driver test
>        🚧 ⚡⚡⚡ IPMItool loop stress test
>        🚧 ⚡⚡⚡ Storage blktests
>        🚧 ⚡⚡⚡ Storage block - filesystem fio test
>        🚧 ⚡⚡⚡ Storage block - queue scheduler test
>        🚧 ⚡⚡⚡ Storage nvme - tcp
>        🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
> 
>     Host 2: https://beaker.engineering.redhat.com/recipes/9115541
>        ✅ Boot test
>        ✅ ACPI table test
>        ✅ ACPI enabled test
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory: fork_mem
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Networking socket: fuzz
>        ✅ Networking: igmp conformance test
>        ✅ Networking route: pmtu
>        ✅ Networking route_func - local
>        ✅ Networking route_func - forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns - transport
>        ✅ Networking ipsec: basic netns - tunnel
>        ✅ Libkcapi AF_ALG test
>        ✅ pciutils: update pci ids test
>        ✅ ALSA PCM loopback test
>        ✅ ALSA Control (mixer) Userspace Element test
>        ✅ storage: SCSI VPD
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ Firmware test suite
>        🚧 ✅ jvm - jcstress tests
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ Ethernet drivers sanity
>        🚧 ✅ Networking firewall: basic netfilter test
>        🚧 ✅ audit: audit testsuite test
>        🚧 ✅ trace: ftrace/tracer
>        🚧 ✅ kdump - kexec_boot
> 
>     Host 3: https://beaker.engineering.redhat.com/recipes/9115727
> 
>        ⚡ Internal infrastructure issues prevented one or more tests (marked
>        with ⚡⚡⚡) from running on this architecture.
>        This is not the fault of the kernel that was tested.
> 
>        ⚡⚡⚡ Boot test
>        ⚡⚡⚡ selinux-policy: serge-testsuite
>        ⚡⚡⚡ storage: software RAID testing
>        ⚡⚡⚡ stress: stress-ng
>        🚧 ⚡⚡⚡ xfstests - ext4
>        🚧 ⚡⚡⚡ xfstests - xfs
>        🚧 ⚡⚡⚡ xfstests - btrfs
>        🚧 ⚡⚡⚡ IPMI driver test
>        🚧 ⚡⚡⚡ IPMItool loop stress test
>        🚧 ⚡⚡⚡ Storage blktests
>        🚧 ⚡⚡⚡ Storage block - filesystem fio test
>        🚧 ⚡⚡⚡ Storage block - queue scheduler test
>        🚧 ⚡⚡⚡ Storage nvme - tcp
>        🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
> 
>     Host 4: https://beaker.engineering.redhat.com/recipes/9115861
>        ✅ Boot test
>        ✅ selinux-policy: serge-testsuite
>        ✅ storage: software RAID testing
>        ✅ stress: stress-ng
>        🚧 ❌ xfstests - ext4
>        🚧 ✅ xfstests - xfs
>        🚧 ✅ xfstests - btrfs
>        🚧 ❌ IPMI driver test
>        🚧 ✅ IPMItool loop stress test
>        🚧 ✅ Storage blktests
>        🚧 ✅ Storage block - filesystem fio test
>        🚧 ✅ Storage block - queue scheduler test
>        🚧 ✅ Storage nvme - tcp
>        🚧 ✅ Storage: swraid mdadm raid_module test
> 
>   ppc64le:
>     Host 1: https://beaker.engineering.redhat.com/recipes/9115549
>        ✅ Boot test
>        ✅ selinux-policy: serge-testsuite
>        ✅ storage: software RAID testing
>        🚧 ❌ xfstests - ext4
>        🚧 ✅ xfstests - xfs
>        🚧 ✅ xfstests - btrfs
>        🚧 ✅ IPMI driver test
>        🚧 ✅ IPMItool loop stress test
>        🚧 ✅ Storage blktests
>        🚧 ✅ Storage block - filesystem fio test
>        🚧 ✅ Storage block - queue scheduler test
>        🚧 ✅ Storage nvme - tcp
>        🚧 ✅ Storage: swraid mdadm raid_module test
> 
>     Host 2: https://beaker.engineering.redhat.com/recipes/9115548
>        ✅ Boot test
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory: fork_mem
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Networking socket: fuzz
>        ✅ Networking route: pmtu
>        ✅ Networking route_func - local
>        ✅ Networking route_func - forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns - tunnel
>        ✅ Libkcapi AF_ALG test
>        ✅ pciutils: update pci ids test
>        ✅ ALSA PCM loopback test
>        ✅ ALSA Control (mixer) Userspace Element test
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ jvm - jcstress tests
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ Ethernet drivers sanity
>        🚧 ✅ Networking firewall: basic netfilter test
>        🚧 ✅ audit: audit testsuite test
>        🚧 ✅ trace: ftrace/tracer
> 
>   s390x:
>     Host 1: https://beaker.engineering.redhat.com/recipes/9115547
>        ✅ Boot test
>        ✅ selinux-policy: serge-testsuite
>        ✅ stress: stress-ng
>        🚧 ✅ Storage blktests
>        🚧 ❌ Storage nvme - tcp
>        🚧 ✅ Storage: swraid mdadm raid_module test
> 
>     Host 2: https://beaker.engineering.redhat.com/recipes/9115546
>        ✅ Boot test
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory: fork_mem
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        ✅ Networking route: pmtu
>        ✅ Networking route_func - local
>        ✅ Networking route_func - forward
>        ✅ Networking TCP: keepalive test
>        ✅ Networking UDP: socket
>        ✅ Networking tunnel: geneve basic test
>        ✅ Networking tunnel: gre basic
>        ✅ L2TP basic test
>        ✅ Networking tunnel: vxlan basic
>        ✅ Networking ipsec: basic netns - transport
>        ✅ Networking ipsec: basic netns - tunnel
>        ✅ Libkcapi AF_ALG test
>        🚧 ✅ CIFS Connectathon
>        🚧 ✅ POSIX pjd-fstest suites
>        🚧 ✅ jvm - jcstress tests
>        🚧 ✅ Memory function: kaslr
>        🚧 ✅ Ethernet drivers sanity
>        🚧 ✅ Networking firewall: basic netfilter test
>        🚧 ✅ audit: audit testsuite test
>        🚧 ✅ trace: ftrace/tracer
> 
>   x86_64:
>     Host 1: https://beaker.engineering.redhat.com/recipes/9115543
>        ✅ Boot test
>        ✅ ACPI table test
>        ✅ LTP
>        ✅ Loopdev Sanity
>        ✅ Memory: fork_mem
>        ✅ Memory function: memfd_create
>        ✅ AMTU (Abstract Machine Test Utility)
>        ✅ Networking bridge: sanity
>        💥 Networking socket: fuzz
>        ⚡⚡⚡ Networking: igmp conformance test
>        ⚡⚡⚡ Networking route: pmtu
>        ⚡⚡⚡ Networking route_func - local
>        ⚡⚡⚡ Networking route_func - forward
>        ⚡⚡⚡ Networking TCP: keepalive test
>        ⚡⚡⚡ Networking UDP: socket
>        ⚡⚡⚡ Networking tunnel: geneve basic test
>        ⚡⚡⚡ Networking tunnel: gre basic
>        ⚡⚡⚡ L2TP basic test
>        ⚡⚡⚡ Networking tunnel: vxlan basic
>        ⚡⚡⚡ Networking ipsec: basic netns - transport
>        ⚡⚡⚡ Networking ipsec: basic netns - tunnel
>        ⚡⚡⚡ Libkcapi AF_ALG test
>        ⚡⚡⚡ pciutils: sanity smoke test
>        ⚡⚡⚡ pciutils: update pci ids test
>        ⚡⚡⚡ ALSA PCM loopback test
>        ⚡⚡⚡ ALSA Control (mixer) Userspace Element test
>        ⚡⚡⚡ storage: SCSI VPD
>        🚧 ⚡⚡⚡ CIFS Connectathon
>        🚧 ⚡⚡⚡ POSIX pjd-fstest suites
>        🚧 ⚡⚡⚡ Firmware test suite
>        🚧 ⚡⚡⚡ jvm - jcstress tests
>        🚧 ⚡⚡⚡ Memory function: kaslr
>        🚧 ⚡⚡⚡ Ethernet drivers sanity
>        🚧 ⚡⚡⚡ Networking firewall: basic netfilter test
>        🚧 ⚡⚡⚡ audit: audit testsuite test
>        🚧 ⚡⚡⚡ trace: ftrace/tracer
>        🚧 ⚡⚡⚡ kdump - kexec_boot
> 
>     Host 2: https://beaker.engineering.redhat.com/recipes/9115545
> 
>        ⚡ Internal infrastructure issues prevented one or more tests (marked
>        with ⚡⚡⚡) from running on this architecture.
>        This is not the fault of the kernel that was tested.
> 
>        ⚡⚡⚡ Boot test
>        ⚡⚡⚡ selinux-policy: serge-testsuite
>        ⚡⚡⚡ storage: software RAID testing
>        ⚡⚡⚡ stress: stress-ng
>        🚧 ⚡⚡⚡ CPU: Frequency Driver Test
>        🚧 ⚡⚡⚡ CPU: Idle Test
>        🚧 ⚡⚡⚡ xfstests - ext4
>        🚧 ⚡⚡⚡ xfstests - xfs
>        🚧 ⚡⚡⚡ xfstests - btrfs
>        🚧 ⚡⚡⚡ IPMI driver test
>        🚧 ⚡⚡⚡ IPMItool loop stress test
>        🚧 ⚡⚡⚡ power-management: cpupower/sanity test
>        🚧 ⚡⚡⚡ Storage blktests
>        🚧 ⚡⚡⚡ Storage block - filesystem fio test
>        🚧 ⚡⚡⚡ Storage block - queue scheduler test
>        🚧 ⚡⚡⚡ Storage nvme - tcp
>        🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
> 
>     Host 3: https://beaker.engineering.redhat.com/recipes/9115544
>        ✅ Boot test
>        🚧 ✅ kdump - sysrq-c
>        🚧 ✅ kdump - file-load
> 
>     Host 4: https://beaker.engineering.redhat.com/recipes/9115908
> 
>        ⚡ Internal infrastructure issues prevented one or more tests (marked
>        with ⚡⚡⚡) from running on this architecture.
>        This is not the fault of the kernel that was tested.
> 
>        ✅ Boot test
>        ✅ selinux-policy: serge-testsuite
>        ✅ storage: software RAID testing
>        ⚡⚡⚡ stress: stress-ng
>        🚧 ⚡⚡⚡ CPU: Frequency Driver Test
>        🚧 ⚡⚡⚡ CPU: Idle Test
>        🚧 ⚡⚡⚡ xfstests - ext4
>        🚧 ⚡⚡⚡ xfstests - xfs
>        🚧 ⚡⚡⚡ xfstests - btrfs
>        🚧 ⚡⚡⚡ IPMI driver test
>        🚧 ⚡⚡⚡ IPMItool loop stress test
>        🚧 ⚡⚡⚡ power-management: cpupower/sanity test
>        🚧 ⚡⚡⚡ Storage blktests
>        🚧 ⚡⚡⚡ Storage block - filesystem fio test
>        🚧 ⚡⚡⚡ Storage block - queue scheduler test
>        🚧 ⚡⚡⚡ Storage nvme - tcp
>        🚧 ⚡⚡⚡ Storage: swraid mdadm raid_module test
> 
>     Host 5: https://beaker.engineering.redhat.com/recipes/9116655
>        ✅ Boot test
>        ✅ selinux-policy: serge-testsuite
>        ✅ storage: software RAID testing
>        ✅ stress: stress-ng
>        🚧 ❌ CPU: Frequency Driver Test
>        🚧 ✅ CPU: Idle Test
>        🚧 ❌ xfstests - ext4
>        🚧 ✅ xfstests - xfs
>        🚧 ✅ xfstests - btrfs
>        🚧 ❌ IPMI driver test
>        🚧 ✅ IPMItool loop stress test
>        🚧 ✅ Storage blktests
>        🚧 ✅ Storage block - filesystem fio test
>        🚧 ✅ Storage block - queue scheduler test
>        🚧 ✅ Storage nvme - tcp
>        🚧 ✅ Storage: swraid mdadm raid_module test
> 
>   Test sources: https://gitlab.com/cki-project/kernel-tests
>     💚 Pull requests are welcome for new tests or improvements to existing
>     tests!
> 
> Aborted tests
> -------------
> Tests that didn't complete running successfully are marked with ⚡⚡⚡.
> If this was caused by an infrastructure issue, we try to mark that
> explicitly in the report.
> 
> Waived tests
> ------------
> If the test run included waived tests, they are marked with 🚧. Such tests
> are
> executed but their results are not taken into account. Tests are waived when
> their results are not reliable enough, e.g. when they're just introduced or
> are
> being fixed.
> 
> Testing timeout
> ---------------
> We aim to provide a report within reasonable timeframe. Tests that haven't
> finished running yet are marked with ⏱.
> 
>