lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [day] [month] [year] [list]
Message-ID: <b75bec0d-e083-46e9-96fe-47abbf089cbd@uwaterloo.ca>
Date: Fri, 1 Nov 2024 17:46:03 -0400
From: Martin Karsten <mkarsten@...terloo.ca>
To: "Samudrala, Sridhar" <sridhar.samudrala@...el.com>,
 Joe Damato <jdamato@...tly.com>, netdev@...r.kernel.org, pabeni@...hat.com,
 namangulati@...gle.com, edumazet@...gle.com, amritha.nambiar@...el.com,
 sdf@...ichev.me, peter@...eblog.net, m2shafiei@...terloo.ca,
 bjorn@...osinc.com, hch@...radead.org, willy@...radead.org,
 willemdebruijn.kernel@...il.com, skhawaja@...gle.com, kuba@...nel.org,
 Bagas Sanjaya <bagasdotme@...il.com>, "David S. Miller"
 <davem@...emloft.net>, Simon Horman <horms@...nel.org>,
 Jonathan Corbet <corbet@....net>,
 "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
 open list <linux-kernel@...r.kernel.org>,
 "open list:BPF [MISC] :Keyword:(?:b|_)bpf(?:b|_)" <bpf@...r.kernel.org>
Subject: Re: [PATCH net-next v3 7/7] docs: networking: Describe irq suspension

On 2024-11-01 17:01, Samudrala, Sridhar wrote:
> 
> 
> On 10/31/2024 11:39 PM, Joe Damato wrote:
>> On Thu, Oct 31, 2024 at 10:47:05PM -0500, Samudrala, Sridhar wrote:
>>>
>>>
>>> On 10/31/2024 7:48 PM, Joe Damato wrote:
>>>> Describe irq suspension, the epoll ioctls, and the tradeoffs of using
>>>> different gro_flush_timeout values.

[...]

>>>> +To use this mechanism:
>>>> +
>>>> +  1. The per-NAPI config parameter ``irq_suspend_timeout`` should 
>>>> be set to the
>>>> +     maximum time (in nanoseconds) the application can have its IRQs
>>>> +     suspended. This is done using netlink, as described above. 
>>>> This timeout
>>>> +     serves as a safety mechanism to restart IRQ driver interrupt 
>>>> processing if
>>>> +     the application has stalled. This value should be chosen so 
>>>> that it covers
>>>> +     the amount of time the user application needs to process data 
>>>> from its
>>>> +     call to epoll_wait, noting that applications can control how 
>>>> much data
>>>> +     they retrieve by setting ``max_events`` when calling epoll_wait.
>>>> +
>>>> +  2. The sysfs parameter or per-NAPI config parameters 
>>>> ``gro_flush_timeout``
>>>> +     and ``napi_defer_hard_irqs`` can be set to low values. They 
>>>> will be used
>>>> +     to defer IRQs after busy poll has found no data.
>>>
>>> Is it required to set gro_flush_timeout and napi_defer_hard_irqs when
>>> irq_suspend_timeout is set? Doesn't it override any smaller
>>> gro_flush_timeout value?
>>
>> It is not required to use gro_flush_timeout or napi_defer_hard_irqs,
>> but if they are set they will take over when epoll finds no events.
>> Their usage is recommended. See the Usage section of the cover
>> letter for details.
>>
>> While gro_flush_timeout and napi_defer_hard_irqs are not strictly
>> required, it is difficult for the polling-based packet delivery loop
>> to gain control over packet delivery.
>>
>> Please see a previous email about this from the RFC for more
>> details:
>>
>> https://lore.kernel.org/netdev/2bb121dd-3dcd-4142- 
>> ab87-02ccf4afd469@...terloo.ca/
> 
> OK. Thanks for the clarification.
>>
>> In the cover letter, you can note the difference in performance when
>> gro_flush_timeout is set to different values. Note the explanation
>> of suspendX; each suspend case is testing a different
>> gro_flush_timeout.
> 
> May be you can also include a test scenario in your perf results  where 
> gro_flush_timeout and napi_defer_hard_irqs are not set to show that a 
> non-zero value of gro_flush_timeout and napi_defer_hard_irqs is 
> recommended when using irq_suspend_timeout.

Thanks for your feedback. We've updated the cover letter as well as the 
kernel documentation to explain this in more detail and to illustrate 
why the parameter usage is recommended. We ran experiments with these 
parameters set to zero and the results are as expected and essentially 
the same as the base case, i.e., irq_suspend_timeout does not have an 
effect in this case.

Thanks,
Martin



> 
>>
>> Let us know if you have any other questions; both Martin and I are
>> happy to help or further explain anything that is not clear.
>>
> 


Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ