netdev - Re: [PATCH net-next v2] net: stop napi kthreads when THREADED napi is disabled

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAAywjhScfevLxYho-wxU6WNF+0VpwngW8MzZjpx1HQ83NTXUDw@mail.gmail.com>
Date: Wed, 21 May 2025 15:50:19 -0700
From: Samiullah Khawaja <skhawaja@...gle.com>
To: Jakub Kicinski <kuba@...nel.org>
Cc: Wei Wang <weiwan@...gle.com>, "David S . Miller" <davem@...emloft.net>, 
	Eric Dumazet <edumazet@...gle.com>, Paolo Abeni <pabeni@...hat.com>, almasrymina@...gle.com, 
	willemb@...gle.com, jdamato@...tly.com, mkarsten@...terloo.ca, 
	netdev@...r.kernel.org
Subject: Re: [PATCH net-next v2] net: stop napi kthreads when THREADED napi is disabled

On Wed, May 21, 2025 at 3:21 PM Jakub Kicinski <kuba@...nel.org> wrote:
>
> On Wed, 21 May 2025 12:51:33 -0700 Samiullah Khawaja wrote:
> > > This might suffer from the problem you highlighted earlier,
> > > CPU 0 (IRQ)             CPU 1 (NAPI thr)          CPU 2 (config)
> > >
> > >   ____napi_schedule()
> > >     if (test_bit(NAPI_STATE_THREADED))
> > >     if (thread) {
> > >
> > >  kthread_stop()
> > >                               if (state & SCHED_THREADED || !(state & SCHED)) {
> > >                                    state &= ~THREADED;
> > >                               if (try_cmp_xchg())
> > >                                    break
> > >
> > >        set_bit(NAPI_STATE_SCHED_THREADED)
> > >        wake_up_process(thread);
>
> This got a bit line wrapped for me so can't judge :(
Yep, sorry about that. But I think we are on the same page.
>
> > > This would happen without the try_cmp_xchg logic that I added in my
> > > patch in the __napi_schedule (in the fast path). __napi_schedule would
> > > have to make sure that the kthread is not stopping while it is trying
> > > to do SCHED. This is similar to the logic we have in
> > > napi_schedule_prep that handles the STATE_DISABLE, STATE_SCHED and
> > > STATE_MISSED scenarios. Also if it falls back to normal softirq, it
> > > needs to make sure that the kthread is not polling at the same time.
> > Discard this as the SCHED would be set in napi_schedule_prepare before
> > __napi_schedule is called in IRQ, so try_cmp_xchg would return false.
> > I think if the thread stops if the napi is idle(SCHED is not) set then
> > it should do. This should make sure any pending SCHED_THREADED are
> > also done. The existing logic in napi_schedulle_prep should handle all
> > the cases.
>
> I think we're on the same page. We're clearing the THREADED bit
> (not SCHED_THREADED). Only napi_schedule() path looks at that bit,
> after setting SCHED. So if we cmpxchg on a state where SCHED was
> clear - we can't race with anything that cares about THREADED bit.
+1

Yes, the logic in napi_sched_prep should provide enough mutual
exclusion. The only problem is that we have to do a poll if
SCHED_THREADED is set. But if we unset THREADED (using the cmpxchg
logic) then do a poll, the final poll should unset SCHED_THREADED also
and would be safe to stop. I am going to move this loop to the
napi_thread_poll function so it is a little bit clean.
>
> Just to be clear - the stopping of the thread has to be after the
> proposed loop, so kthread_should_stop() does not come into play.
Wait, the thread will unset STATE_THREADED if kthread_should_stop is
true. right? Otherwise how would thread know that it has to unset the
bit and stop?

As I understand, we should be doing something like following:

while (true) {
   state = READ_ONCE()
   can_stop = false;

   if (kthread_should_stop) {
       if (SCHED_THREADED || !SCHED) {
           state &= !THREADED
       } else {
           msleep(1);
           continue;
       }

        if (try_cmpxchg) {
            can_stop = true;
            if (!SCHED_THREADED)
                break;
        }
   }

   if (SCHED_THREADED)
       poll()

   if (can_stop))
       break;
}

dev_set_threaded() {
    ...
   if (!threaded)
      kthread_stop(thread)
}

>
> And FWIW my understanding is that we don't need any barriers on the
> fast path (SCHED vs checking THREADED) because memory ordering is
> a thing which exists only between distinct memory words.