netdev - [TEST] Flake report

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date: Thu, 9 May 2024 16:09:58 -0700
From: Jakub Kicinski <kuba@...nel.org>
To: Florian Westphal <fw@...len.de>, Simon Horman <horms@...nel.org>,
 Hangbin Liu <liuhangbin@...il.com>, Jaehee Park <jhpark1013@...il.com>,
 Petr Machata <petrm@...dia.com>, Nikolay Aleksandrov <razor@...ckwall.org>,
 Ido Schimmel <idosch@...dia.com>, Davide Caratti <dcaratti@...hat.com>,
 Matthieu Baerts <matttbe@...nel.org>
Cc: netdev@...r.kernel.org
Subject: [TEST] Flake report

Hi!

Feels like the efforts to get rid of flaky tests have slowed down a bit,
so I thought I'd poke people..

Here's the full list:
https://netdev.bots.linux.dev/flakes.html?min-flip=0&pw-y=0
click on test name to get the list of runs and links to outputs.

As a reminder please see these instructions for repro:
https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style

I'll try to tag folks who touched the tests most recently, but please
don't hesitate to chime in.

net
---

arp-ndisc-untracked-subnets-sh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To: Jaehee Park <jhpark1013@...il.com>
Cc: Hangbin Liu <liuhangbin@...il.com>

Times out on debug kernels, passes on non-debug.
This is a real timeout, eats full 7200 seconds.

xfrm-policy-sh
~~~~~~~~~~~~~~
To: Hangbin Liu <liuhangbin@...il.com>

Times out on debug kernels, passed on non-debug,
This is a "inactivity" timeout, test doesn't print anything
for 900 seconds so the runner kills it. We can bump the timeout
but not printing for 15min is bad..

cmsg-time-sh
~~~~~~~~~~~~
To: Jakub Kicinski <kuba@...nel.org> (forgot I wrote this :D)

Fails randomly.

pmtu-sh
~~~~~~~
To: Simon Horman <horms@...nel.org>

Skipped because it wants full OVS tooling.

forwarding
----------

sch-tbf-ets-sh, sch-tbf-prio-sh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To: Petr Machata <petrm@...dia.com>

These fail way too often on non-debug kernels :(
Perhaps we can extend the lower bound?

bridge-igmp-sh, bridge-mld-sh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To: Nikolay Aleksandrov <razor@...ckwall.org>
Cc: Ido Schimmel <idosch@...dia.com>

On debug kernels it always fails with:

# TEST: IGMPv3 group 239.10.10.10 exclude timeout                     [FAIL]
# Entry 192.0.2.21 has blocked flag failed

For MLD:

# TEST: MLDv2 group ff02::cc exclude timeout                          [FAIL]
# Entry 2001:db8:1::21 has blocked flag failed

vxlan-bridge-1d-sh
~~~~~~~~~~~~~~~~~~
To: Ido Schimmel <idosch@...dia.com>
Cc: Petr Machata <petrm@...dia.com>

Flake fails almost always, with some form of "Expected to capture 0
packets, got $X"

mirror-gre-lag-lacp-sh
~~~~~~~~~~~~~~~~~~~~~~
To: Petr Machata <petrm@...dia.com>

Often fails on debug with:

# TEST: mirror to gretap: LAG first slave (skip_hw)                   [FAIL]
# Expected to capture 10 packets, got 13.

mirror-gre-vlan-bridge-1q-sh, mirror-gre-bridge-1d-vlan-sh
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
To: Petr Machata <petrm@...dia.com>

Same kind of failure as above but less often and both on debug and non-debug.

tc-actions-sh
~~~~~~~~~~~~~
To: Davide Caratti <dcaratti@...hat.com>

It triggers a random unhandled interrupt, somehow (look at stderr).
It's the only test that does that.

mptcp
-----
To: Matthieu Baerts <matttbe@...nel.org>

simult-flows-sh is still quite flaky :(

nf
--
To: Florian Westphal <fw@...len.de>

These are skipped because of some compatibility issues:

 nft-flowtable-sh, bridge-brouter-sh, nft-audit-sh

Please LMK if I need to update the CLI tooling. 
Or is this missing kernel config?