Message-ID: <CAMuHMdV2SLyrTs5MJKwAL2-jVLpd=TP+bMMnuEj-Ump0oyLjJA@mail.gmail.com>
Date: Tue, 27 Jan 2026 13:02:44 +0100
From: Geert Uytterhoeven <geert@...ux-m68k.org>
To: Tommaso Merciai <tommaso.merciai.xr@...renesas.com>
Cc: Marek Szyprowski <m.szyprowski@...sung.com>, Xin Zhao <jackzxcui1989@....com>, 
	gregkh@...uxfoundation.org, jirislaby@...nel.org, tj@...nel.org, 
	hch@...radead.org, linux-kernel@...r.kernel.org, linux-serial@...r.kernel.org, 
	Linux-Renesas <linux-renesas-soc@...r.kernel.org>
Subject: Re: [PATCH v8] tty: tty_port: add workqueue to flip TTY buffer

On Tue, 27 Jan 2026 at 12:10, Tommaso Merciai
<tommaso.merciai.xr@...renesas.com> wrote:
> On Tue, Jan 27, 2026 at 11:34:32AM +0100, Marek Szyprowski wrote:
> > On 23.12.2025 04:48, Xin Zhao wrote:
> > > On the embedded platform, certain critical data, such as IMU data, is
> > > transmitted through UART. The tty_flip_buffer_push() interface in the TTY
> > > layer uses system_dfl_wq to handle the flipping of the TTY buffer.
> > > Although the unbound workqueue can create new threads on demand and wake
> > > up the kworker thread on an idle CPU, it may be preempted by real-time
> > > tasks or other high-prio tasks.
> > >
> > > flush_to_ldisc() needs to wake up the relevant data handle thread. When
> > > executing __wake_up_common_lock(), it calls spin_lock_irqsave(), which
> > > does not disable preemption but disables migration in RT-Linux. This
> > > prevents the kworker thread from being migrated to other cores by CPU's
> > > balancing logic, resulting in long delays. The call trace is as follows:
> > >      __wake_up_common_lock
> > >      __wake_up
> > >      ep_poll_callback
> > >      __wake_up_common
> > >      __wake_up_common_lock
> > >      __wake_up
> > >      n_tty_receive_buf_common
> > >      n_tty_receive_buf2
> > >      tty_ldisc_receive_buf
> > >      tty_port_default_receive_buf
> > >      flush_to_ldisc
> > >
> > > In our system, the processing interval for each frame of IMU data
> > > transmitted via UART can experience significant jitter due to this issue.
> > > Instead of the expected 10 to 15 ms frame processing interval, we see
> > > spikes up to 30 to 35 ms. Moreover, in just one or two hours, there can
> > > be 2 to 3 occurrences of such high jitter, which is quite frequent. This
> > > jitter exceeds the software's tolerable limit of 20 ms.
> > >
> > > Introduce flip_wq in tty_port which can be set by tty_port_link_wq() or as
> > > default linked to default workqueue allocated when tty_register_driver().
> > > The default workqueue is allocated with flag WQ_SYSFS, so that cpumask and
> > > nice can be set dynamically. The execution timing of tty_port_link_wq() is
> > > not clearly restricted. The newly added function tty_port_link_driver_wq()
> > > checks whether the flip_wq of the tty_port has already been assigned when
> > > linking the default tty_driver's workqueue to the port. After the user has
> > > set a custom workqueue for a certain tty_port using tty_port_link_wq(), the
> > > system will only use this custom workqueue, even if tty_driver does not
> > > have %TTY_DRIVER_CUSTOM_WORKQUEUE flag.
> > >
> > > Introduce %TTY_DRIVER_CUSTOM_WORKQUEUE flag meaning not to create the
> > > default single tty_driver workqueue. Two reasons why need to introduce the
> > > %TTY_DRIVER_CUSTOM_WORKQUEUE flag:
> > > 1. If the WQ_SYSFS parameter is enabled, workqueue_sysfs_register() will
> > > fail when trying to create a workqueue with the same name. The pty is an
> > > example of this; if both CONFIG_LEGACY_PTYS and CONFIG_UNIX98_PTYS are
> > > enabled, the call to tty_register_driver() in unix98_pty_init() will fail.
> > > 2. Different tty ports may be used for different tasks, which may require
> > > separate core binding control via workqueues. In this case, the workqueue
> > > created by default in the tty driver is unnecessary. Enabling this flag
> > > prevents the creation of this redundant workqueue.
> > >
> > > After applying this patch, we can set the related UART TTY flip buffer
> > > workqueue by sysfs. We set the cpumask to CPU cores associated with the
> > > IMU tasks, and set the nice to -20. Testing has shown significant
> > > improvement in the previously described issue, with almost no stuttering
> > > occurring anymore.
> > >
> > > Signed-off-by: Xin Zhao <jackzxcui1989@....com>
> >
> > This patch landed in linux-next as commit d000422a46aa ("tty: tty_port:
> > add workqueue to flip TTY buffer"). In my tests I found that it causes
> > some regressions, see the comments in the code below.
>
> Same here, testing on RZ/G3E looks like s2idle is broken:

> [  185.237717] Call trace:
> [  185.240176]  __queue_work+0x20/0x474 (P)
> [  185.244141]  queue_work_on+0x8c/0xa8
> [  185.247753]  tty_flip_buffer_push+0x2c/0x38

Lucky you, there is a hint to tty in your trace ;-)

I see a similar crash during boot on koelsch (R-Car M2-W), and a
lock-up during boot on salvator-xs (R-Car H3 ES2.0), with either no
output or an rcu stall:

    rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
    rcu:     1-...!: (0 ticks this GP) idle=1fe8/0/0x0 softirq=85/85
fqs=1 (false positive?)
    rcu:     2-...!: (1 ticks this GP) idle=1c78/0/0x0 softirq=77/77
fqs=1 (false positive?)
    rcu:     6-...!: (0 ticks this GP) idle=07b8/0/0x0 softirq=9/9
fqs=1 (false positive?)
    rcu:     (detected by 3, t=5260 jiffies, g=-1015, q=274 ncpus=8)
    Sending NMI from CPU 3 to CPUs 1:
    Sending NMI from CPU 3 to CPUs 2:
    Sending NMI from CPU 3 to CPUs 6:
    rcu: rcu_preempt kthread timer wakeup didn't happen for 12771
jiffies! g-1015 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x200
    rcu:     Possible timer handling issue on cpu=6 timer-softirq=1
    rcu: rcu_preempt kthread starved for 12780 jiffies! g-1015 f0x0
RCU_GP_WAIT_FQS(5) ->state=0x200 ->cpu=6
    rcu:     Unless rcu_preempt kthread gets sufficient CPU time, OOM
is now expected behavior.
    rcu: RCU grace-period kthread stack dump:
    task:rcu_preempt     state:R stack:0     pid:15    tgid:15
ppid:2      task_flags:0x208040 flags:0x00000010
    Call trace:
     __switch_to+0xcc/0x100 (T)
     __schedule+0x368/0xc00
     schedule+0x30/0x100
     schedule_timeout+0x80/0xf8
     rcu_gp_fqs_loop+0xfc/0x418
     rcu_gp_kthread+0xe0/0xf4
     kthread+0x128/0x1e0
     ret_from_fork+0x10/0x20

Reverting commit d000422a46aad322 ("tty: tty_port: add workqueue to
flip TTY buffer") in tty-next fixes both.

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@...ux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Powered by blists - more mailing lists