netdev - [PATCH net-next 2/2] selftests: net: ksft: support marking tests as disruptive

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240729221042.2700882-2-sdf@fomichev.me>
Date: Mon, 29 Jul 2024 15:10:42 -0700
From: Stanislav Fomichev <sdf@...ichev.me>
To: netdev@...r.kernel.org
Cc: davem@...emloft.net,
	edumazet@...gle.com,
	kuba@...nel.org,
	pabeni@...hat.com,
	Shuah Khan <shuah@...nel.org>,
	Joe Damato <jdamato@...tly.com>,
	Petr Machata <petrm@...dia.com>,
	linux-kselftest@...r.kernel.org
Subject: [PATCH net-next 2/2] selftests: net: ksft: support marking tests as disruptive

(not sure we want this, but just throwing it out there..)

Add new @ksft_disruptive decorator to mark the tests that might
be disruptive to the system. Depending on how well the previous
test works in the CI we might want to disable disruptive tests
by default and only let the developers run them manually.

In the future we can add similar decorators to, for example, avoid
running slow tests all the time. And/or have some option to run
only 'fast' tests for some sort of smoke test scenario.

$ ./stats.py --skip-disruptive
KTAP version 1
1..5
ok 1 stats.check_pause
ok 2 stats.check_fec
ok 3 stats.pkt_byte_sum
ok 4 stats.qstat_by_ifindex
ok 5 stats.check_down # SKIP marked as disruptive
\# Totals: pass:4 fail:0 xfail:0 xpass:0 skip:1 error:0

Signed-off-by: Stanislav Fomichev <sdf@...ichev.me>
--
Cc: Shuah Khan <shuah@...nel.org>
Cc: Joe Damato <jdamato@...tly.com>
Cc: Petr Machata <petrm@...dia.com>
Cc: linux-kselftest@...r.kernel.org
---
 tools/testing/selftests/drivers/net/stats.py |  2 ++
 tools/testing/selftests/net/lib/py/ksft.py   | 22 ++++++++++++++++++++
 2 files changed, 24 insertions(+)

diff --git a/tools/testing/selftests/drivers/net/stats.py b/tools/testing/selftests/drivers/net/stats.py
index 6f8bef379565..04508894ef9c 100755
--- a/tools/testing/selftests/drivers/net/stats.py
+++ b/tools/testing/selftests/drivers/net/stats.py
@@ -3,6 +3,7 @@
 
 from lib.py import ksft_run, ksft_exit, ksft_pr
 from lib.py import ksft_ge, ksft_eq, ksft_in, ksft_true, ksft_raises, KsftSkipEx, KsftXfailEx
+from lib.py import ksft_disruptive
 from lib.py import EthtoolFamily, NetdevFamily, RtnlFamily, NlError
 from lib.py import NetDrvEnv
 from lib.py import ip
@@ -134,6 +135,7 @@ rtnl = RtnlFamily()
     ksft_eq(cm.exception.nl_msg.extack['bad-attr'], '.ifindex')
 
 
+@...t_disruptive
 def check_down(cfg) -> None:
     try:
         qstat = netfam.qstats_get({"ifindex": cfg.ifindex}, dump=True)
diff --git a/tools/testing/selftests/net/lib/py/ksft.py b/tools/testing/selftests/net/lib/py/ksft.py
index f26c20df9db4..f6545a45942a 100644
--- a/tools/testing/selftests/net/lib/py/ksft.py
+++ b/tools/testing/selftests/net/lib/py/ksft.py
@@ -1,6 +1,8 @@
 # SPDX-License-Identifier: GPL-2.0
 
+import argparse
 import builtins
+import functools
 import inspect
 import sys
 import time
@@ -10,6 +12,7 @@ from .utils import global_defer_queue
 
 KSFT_RESULT = None
 KSFT_RESULT_ALL = True
+KSFT_ARGS = None
 
 
 class KsftFailEx(Exception):
@@ -127,7 +130,26 @@ KSFT_RESULT_ALL = True
             KSFT_RESULT = False
 
 
+def ksft_disruptive(func):
+    """
+    Decorator that marks the test as disruptive. Disruptive tests
+    can be skipped by adding --skip-disruptive argument.
+    """
+
+    @functools.wraps(func)
+    def wrapper(*args, **kwargs):
+        if KSFT_ARGS.skip_disruptive:
+            raise KsftSkipEx(f"marked as disruptive")
+        return func(*args, **kwargs)
+    return wrapper
+
+
 def ksft_run(cases=None, globs=None, case_pfx=None, args=()):
+    parser = argparse.ArgumentParser()
+    parser.add_argument('--skip-disruptive', default=False, action='store_true', help='skip tests that might be disruptive (e.g. restart the interface)')
+    global KSFT_ARGS
+    KSFT_ARGS = parser.parse_args()
+
     cases = cases or []
 
     if globs and case_pfx:
-- 
2.45.2