netdev - Re: TC: selftests: current timeout (45s) is too low

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Wed, 12 Jul 2023 10:43:35 -0300
From: Pedro Tammela <pctammela@...atatu.com>
To: Matthieu Baerts <matthieu.baerts@...sares.net>,
 Jamal Hadi Salim <jhs@...atatu.com>, Cong Wang <xiyou.wangcong@...il.com>,
 Jiri Pirko <jiri@...nulli.us>
Cc: netdev <netdev@...r.kernel.org>, Anders Roxell
 <anders.roxell@...aro.org>, Davide Caratti <dcaratti@...hat.com>
Subject: Re: TC: selftests: current timeout (45s) is too low

Hi Matthieu,

I have been involved in tdc for a while now, here are my comments.

On 12/07/2023 06:47, Matthieu Baerts wrote:
> Hi Jamal, Cong, Jiri,
> 
> When looking for something else [1] in LKFT reports [2], I noticed that
> the TC selftest ended with a timeout error:
> 
>    not ok 1 selftests: tc-testing: tdc.sh # TIMEOUT 45 seconds
> 
> The timeout has been introduced 3 years ago:
> 
>    852c8cbf34d3 ("selftests/kselftest/runner.sh: Add 45 second timeout
> per test")
> 
> Recently, a new option has been introduced to override the value when
> executing the code:
> 
>    f6a01213e3f8 ("selftests: allow runners to override the timeout")
> 
> But I guess it is still better to set a higher default value for TC
> tests. This is easy to fix by simply adding "timeout=<seconds>" in a
> "settings" file in 'tc-testing' directory, e.g.
> 
>    echo timeout=1200 > tools/testing/selftests/tc-testing/settings
> 
> I'm sending this email instead of a patch because I don't know which
> value makes sense. I guess you know how long the tests can take in a
> (very) slow environment and you might want to avoid this timeout error.

I believe a timeout between 5-10 to minutes should cover the entire suite
> 
> I also noticed most of the tests were skipped [2], probably because
> something is missing in the test environment? Do not hesitate to contact
> the lkft team [3], that's certainly easy to fix and it would increase
> the TC test coverage when they are validating all the different kernel
> versions :)

 From the logs it seems like the kernel image is missing the 'ct' 
action. Possibly also missing other actions/tc components, so it seems 
like a kernel config issue.

Interestingly enough the `tdc.sh` script doesn't test the filter and 
infrastructure categories. Perhaps it would be better to let it run the 
entire suite instead of just a few tests. It could probably break some 
other automated testing out there but it will be worth it, specially 
since we have been adding some regression tests lately.

I do know RedHat runs tdc, so let's ask their opinion.
+Davide WDYT?

> 
> Cheers,
> Matt
> 
> [1] The impact of https://github.com/Linaro/test-definitions/pull/446
> [2]
> https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20230711/testrun/18267241/suite/kselftest-tc-testing/test/tc-testing_tdc_sh/log
> [3] lkft@...aro.org