lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Yz1QnjrOoKtOmCOu@feng-clx>
Date:   Wed, 5 Oct 2022 17:38:38 +0800
From:   Feng Tang <feng.tang@...el.com>
To:     Naresh Kamboju <naresh.kamboju@...aro.org>
CC:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        <linux-kernel@...r.kernel.org>, <stable@...r.kernel.org>,
        <torvalds@...ux-foundation.org>, <akpm@...ux-foundation.org>,
        <linux@...ck-us.net>, <shuah@...nel.org>, <patches@...nelci.org>,
        <lkft-triage@...ts.linaro.org>, <pavel@...x.de>,
        <jonathanh@...dia.com>, <f.fainelli@...il.com>,
        <sudipm.mukherjee@...il.com>, <srw@...dewatkins.net>,
        Hyeonggon Yoo <42.hyeyoo@...il.com>,
        Waiman Long <longman@...hat.com>,
        Vlastimil Babka <vbabka@...e.cz>
Subject: Re: [PATCH 5.19 000/101] 5.19.13-rc1 review

On Tue, Oct 04, 2022 at 12:18:05PM +0530, Naresh Kamboju wrote:
> On Mon, 3 Oct 2022 at 12:43, Greg Kroah-Hartman
> <gregkh@...uxfoundation.org> wrote:
> >
> > This is the start of the stable review cycle for the 5.19.13 release.
> > There are 101 patches in this series, all will be posted as a response
> > to this one.  If anyone has any issues with these being applied, please
> > let me know.
> >
> > Responses should be made by Wed, 05 Oct 2022 07:07:06 +0000.
> > Anything received after that time might be too late.
> >
> > The whole patch series can be found in one patch at:
> >         https://www.kernel.org/pub/linux/kernel/v5.x/stable-review/patch-5.19.13-rc1.gz
> > or in the git tree and branch at:
> >         git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable-rc.git linux-5.19.y
> > and the diffstat can be found below.
> >
> > thanks,
> >
> > greg k-h
> 
> Results from Linaro's test farm.
> No regressions on arm64, arm, x86_64, and i386.
> 
> Tested-by: Linux Kernel Functional Testing <lkft@...aro.org>
> 
> NOTE:
> 1) Build warning
> 2) Boot warning on qemu-arm64 with KASAN and Kunit test
>    Suspecting one of the recently commits causing this warning and
>    need to bisect to confirm the commit id.
>     mm/slab_common: fix possible double free of kmem_cache
> [ Upstream commit d71608a877362becdc94191f190902fac1e64d35 ]

Hi Naresh Kamboju,

Thanks for the report!

Could you try reverting the commit and re-test it to confirm?

Also could you provide the kernel dmesg of the failure and the
kernel config of the test?

I locally pulled the linux-stable source and used QEMU to test
it with kasan/kfence enabled, but could not reproduce it (I
only have x86 HW at hand).

> 2) Following kernel boot warning noticed on qemu-arm64 with KASAN and
> KUNIT enabled [1]
> 
>      [  177.651182] ------------[ cut here ]------------
>      [  177.652217] kmem_cache_destroy test: Slab cache still has
> objects when called from test_exit+0x28/0x40
>      [  177.654849] WARNING: CPU: 0 PID: 1 at mm/slab_common.c:520
> kmem_cache_destroy+0x1e8/0x20c
>      [  177.666237] Modules linked in:
>      [  177.667325] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G    B
>        5.19.13-rc1 #1
>      [  177.668666] Hardware name: linux,dummy-virt (DT)
>      [  177.669783] pstate: 60400005 (nZCv daif +PAN -UAO -TCO -DIT
> -SSBS BTYPE=--)
>      [  177.671120] pc : kmem_cache_destroy+0x1e8/0x20c
>      [  177.672217] lr : kmem_cache_destroy+0x1e8/0x20c
>      [  177.691598] Call trace:
>      [  177.692165]  kmem_cache_destroy+0x1e8/0x20c
>      [  177.693196]  test_exit+0x28/0x40
>      [  177.694158]  kunit_catch_run_case+0x5c/0x120
>      [  177.695177]  kunit_try_catch_run+0x144/0x26c
>      [  177.696211]  kunit_run_case_catch_errors+0x158/0x1e0
>      [  177.697353]  kunit_run_tests+0x374/0x750
>      [  177.698333]  __kunit_test_suites_init+0x74/0xa0
>      [  177.699386]  kunit_run_all_tests+0x160/0x380
>      [  177.700428]  kernel_init_freeable+0x32c/0x388
>      [  177.701497]  kernel_init+0x2c/0x150
>      [  177.702347]  ret_from_fork+0x10/0x20
>      [  177.703308] ---[ end trace 0000000000000000 ]---
> 
> [1] https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2FcCyacq1SusUcnAfamULqzkdUA

I also tried the reproduce cmmand from the above link: 

tuxrun --runtime podman --device qemu-arm64 --kernel https://builds.tuxbuild.com/2FcCwzbNgR7TlQXzJ0nu32y1CpB/Image.gz --modules https://builds.tuxbuild.com/2FcCwzbNgR7TlQXzJ0nu32y1CpB/modules.tar.xz --rootfs https://storage.lkft.org/rootfs/oe-kirkstone/20220824-114729/juno/lkft-tux-image-juno-20220824120304.rootfs.ext4.gz --parameters SKIPFILE=skipfile-lkft.yaml --image docker.io/lavasoftware/lava-dispatcher:2022.06 --tests kunit --timeouts boot=30

Which also didn't reproduce it, but had some RCU stall problems
(could also be related to the x86 HWs)

[  321.006279] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  321.007281]  ffff0000074c2300: 00 07 fc fc fc fc fc fc fc fc fc fc fc fc fc fc
[  321.009283] rcu:      0-...0: (1 GPs behind) idle=40f/1/0x4000000000000000 softirq=436/437 fqs=5

[  321.024995] rcu: rcu_preempt kthread timer wakeup didn't happen for 4464 jiffies! g-207 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402
[  321.026343] rcu:      Possible timer handling issue on cpu=1 timer-softirq=1426
[  321.027340] rcu: rcu_preempt kthread starved for 4465 jiffies! g-207 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=1
[  321.028517] rcu:      Unless rcu_preempt kthread gets sufficient CPU time, OOM is now expected behavior.
[  321.029488] rcu: RCU grace-period kthread stack dump:
[  321.030251] task:rcu_preempt     state:I stack:    0 pid:   16 ppid:     2 flags:0x00000008
[  321.031434] Call trace:
[  321.031878]  __switch_to+0x140/0x1e0
[  321.032565]  __schedule+0x4f4/0xc74
[  321.033228]  schedule+0x88/0x13c
[  321.033915]  schedule_timeout+0x104/0x2b0
[  321.034646]  rcu_gp_fqs_loop+0x1a0/0x784
[  321.035119]  rcu_gp_kthread+0x278/0x3a0
[  321.035608]  kthread+0x160/0x170
[  339.882465]  ret_from_fork+0x10/0x20
[  339.883898] rcu: Stack dump where RCU GP kthread last ran:

The full .xz log is attched.

Thanks,
Feng

Download attachment "stable-k519-kunit.log.xz" of type "application/x-xz" (37972 bytes)

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ