linux-kernel - Re: 'rcu_preempt detected stalls on CPUs/tasks...' issue of cyclictest on rt-linux

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAJNi4rPjsW5ra=MpX918PjZZN+fAsmd0jzGUWYy6nwuEtz4uPg@mail.gmail.com>
Date: Fri, 26 Jul 2024 15:16:01 +0800
From: richard clark <richard.xnu.clark@...il.com>
To: Neeraj upadhyay <neeraj.iitr10@...il.com>
Cc: paulmck@...nel.org, josh@...htriplett.org, 
	Lai Jiangshan <jiangshanlai@...il.com>, mathieu.desnoyers@...icios.com, 
	Steven Rostedt <rostedt@...dmis.org>, Mark Rutland <mark.rutland@....com>, 
	Linus Torvalds <torvalds@...ux-foundation.org>, rcu@...r.kernel.org, 
	linux-kernel@...r.kernel.org, linux-arm-kernel@...ts.infradead.org, 
	linux-rt-users@...r.kernel.org
Subject: Re: 'rcu_preempt detected stalls on CPUs/tasks...' issue of
 cyclictest on rt-linux

Hi，

On Wed, Jul 10, 2024 at 7:22 PM Neeraj upadhyay <neeraj.iitr10@...il.com> wrote:
>
> Hello Richard,
>
> On Wed, Jul 10, 2024 at 1:56 PM richard clark
> <richard.xnu.clark@...il.com> wrote:
> >
> > Hi,
> > I am running a Ubuntu 20.04.5 LTS on Nvidia Jetson AGX Orin platform
> > with 12-cores as a guestOS, the kernel version is - 6.1.83-rt28.
> > Kernel cmdline is:
> > 'root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 mminit_loglevel=4
> > console=ttyTCU0,115200 console=tty0 firmware_class.path=/etc/firmware
> > fbcon=map:0 net.ifnames=0'
> >
> > The cyclictest command 'cyclictest -Smp99 -H 3000
> > --histfile=orin_idle_hyp_4h.hist -D 4h' will hang randomly during the
> > test, then the minicom console will show below messages:
> > ...
> >
> > [97619.450889] [CPU11-E] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
> > [97619.450894] [CPU11-E] rcu:   1-...!: (0 ticks this GP)
> > idle=dc88/0/0x0 softirq=0/0 fqs=2 (false positive?)
> > [97619.450914] [ CPU1-E] NMI backtrace for cpu 1
> > [97619.451912] [CPU11-E] rcu: rcu_preempt kthread timer wakeup didn't
> > happen for 5251 jiffies! g6029253 f0x0 RCU_GP_WAIT_FQS(5)
> > ->state=0x402
> > [97619.451916] [CPU11-E] rcu:   Possible timer handling issue on cpu=1
> > timer-softirq=342864
>
> This log indicates that jiffies timers are not getting handled on CPU1, due to
> which GP kthread was not woken up. Can you check irq, softirq and timer traces
> on CPU1, to see if the softirqs/timers are getting served on this CPU?
>
Please ignore this issue. As narrow down this is an issue from
homemade hypervisor which unmasks the vtimer ctrl.IMASK bit by
accident, so the vcpu can't get further vtimer interrupt, thus the
issue happens...
Thanks

>
> - Neeraj
>