[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <ZyRbZpCiANaxNNlv@LQ3V64L9R2>
Date: Thu, 31 Oct 2024 21:39:02 -0700
From: Joe Damato <jdamato@...tly.com>
To: "Samudrala, Sridhar" <sridhar.samudrala@...el.com>
Cc: netdev@...r.kernel.org, pabeni@...hat.com, namangulati@...gle.com,
edumazet@...gle.com, amritha.nambiar@...el.com, sdf@...ichev.me,
peter@...eblog.net, m2shafiei@...terloo.ca, bjorn@...osinc.com,
hch@...radead.org, willy@...radead.org,
willemdebruijn.kernel@...il.com, skhawaja@...gle.com,
kuba@...nel.org, Martin Karsten <mkarsten@...terloo.ca>,
Bagas Sanjaya <bagasdotme@...il.com>,
"David S. Miller" <davem@...emloft.net>,
Simon Horman <horms@...nel.org>, Jonathan Corbet <corbet@....net>,
"open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
open list <linux-kernel@...r.kernel.org>,
"open list:BPF [MISC] :Keyword:(?:b|_)bpf(?:b|_)" <bpf@...r.kernel.org>
Subject: Re: [PATCH net-next v3 7/7] docs: networking: Describe irq suspension
On Thu, Oct 31, 2024 at 10:47:05PM -0500, Samudrala, Sridhar wrote:
>
>
> On 10/31/2024 7:48 PM, Joe Damato wrote:
> > Describe irq suspension, the epoll ioctls, and the tradeoffs of using
> > different gro_flush_timeout values.
> >
> > Signed-off-by: Joe Damato <jdamato@...tly.com>
> > Co-developed-by: Martin Karsten <mkarsten@...terloo.ca>
> > Signed-off-by: Martin Karsten <mkarsten@...terloo.ca>
> > Acked-by: Stanislav Fomichev <sdf@...ichev.me>
> > Reviewed-by: Bagas Sanjaya <bagasdotme@...il.com>
> > ---
> <snip>
>
>
> > +
> > +IRQ suspension
> > +--------------
> > +
> > +IRQ suspension is a mechanism wherein device IRQs are masked while epoll
> > +triggers NAPI packet processing.
> > +
> > +While application calls to epoll_wait successfully retrieve events, the kernel will
> > +defer the IRQ suspension timer. If the kernel does not retrieve any events
> > +while busy polling (for example, because network traffic levels subsided), IRQ
> > +suspension is disabled and the IRQ mitigation strategies described above are
> > +engaged.
> > +
> > +This allows users to balance CPU consumption with network processing
> > +efficiency.
> > +
> > +To use this mechanism:
> > +
> > + 1. The per-NAPI config parameter ``irq_suspend_timeout`` should be set to the
> > + maximum time (in nanoseconds) the application can have its IRQs
> > + suspended. This is done using netlink, as described above. This timeout
> > + serves as a safety mechanism to restart IRQ driver interrupt processing if
> > + the application has stalled. This value should be chosen so that it covers
> > + the amount of time the user application needs to process data from its
> > + call to epoll_wait, noting that applications can control how much data
> > + they retrieve by setting ``max_events`` when calling epoll_wait.
> > +
> > + 2. The sysfs parameter or per-NAPI config parameters ``gro_flush_timeout``
> > + and ``napi_defer_hard_irqs`` can be set to low values. They will be used
> > + to defer IRQs after busy poll has found no data.
>
> Is it required to set gro_flush_timeout and napi_defer_hard_irqs when
> irq_suspend_timeout is set? Doesn't it override any smaller
> gro_flush_timeout value?
It is not required to use gro_flush_timeout or napi_defer_hard_irqs,
but if they are set they will take over when epoll finds no events.
Their usage is recommended. See the Usage section of the cover
letter for details.
While gro_flush_timeout and napi_defer_hard_irqs are not strictly
required, it is difficult for the polling-based packet delivery loop
to gain control over packet delivery.
Please see a previous email about this from the RFC for more
details:
https://lore.kernel.org/netdev/2bb121dd-3dcd-4142-ab87-02ccf4afd469@uwaterloo.ca/
In the cover letter, you can note the difference in performance when
gro_flush_timeout is set to different values. Note the explanation
of suspendX; each suspend case is testing a different
gro_flush_timeout.
Let us know if you have any other questions; both Martin and I are
happy to help or further explain anything that is not clear.
Powered by blists - more mailing lists