netdev - [PATCH net-next 1/2] selftest: tc-testing: extend the "skip" property

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <34f902ef86d299f3e750be6a4fbade952cae6f3f.1679312049.git.dcaratti@redhat.com>
Date:   Mon, 20 Mar 2023 12:44:54 +0100
From:   Davide Caratti <dcaratti@...hat.com>
To:     Jamal Hadi Salim <jhs@...atatu.com>,
        Cong Wang <xiyou.wangcong@...il.com>,
        Jiri Pirko <jiri@...nulli.us>,
        Ilya Maximets <i.maximets@....org>
Cc:     netdev@...r.kernel.org
Subject: [PATCH net-next 1/2] selftest: tc-testing: extend the "skip" property

currently, users can skip individual test cases by means of writing

  "skip": "yes"

in the scenario file. Extend this functionality by allowing the execution
of a command, written in the "skip" property for a specific test case. If
such property is present, tdc executes that command and skips the test if
the return value is non-zero.

Signed-off-by: Davide Caratti <dcaratti@...hat.com>
---
 .../creating-testcases/AddingTestCases.txt    |  4 +++-
 tools/testing/selftests/tc-testing/tdc.py     | 21 +++++++++++++------
 2 files changed, 18 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt
index a28571aff0e1..130c49ef8576 100644
--- a/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt
+++ b/tools/testing/selftests/tc-testing/creating-testcases/AddingTestCases.txt
@@ -37,7 +37,9 @@ skip:         A completely optional key, if the corresponding value is "yes"
               then tdc will not execute the test case in question. However,
               this test case will still appear in the results output but
               marked as skipped. This key can be placed anywhere inside the
-              test case at the top level.
+              test case at the top level. It's possible to specify a command
+              in the value of "skip": in this case, the test is skipped when
+              the return value is not zero.
 category:     A list of single-word descriptions covering what the command
               under test is testing. Example: filter, actions, u32, gact, etc.
 setup:        The list of commands required to ensure the command under test
diff --git a/tools/testing/selftests/tc-testing/tdc.py b/tools/testing/selftests/tc-testing/tdc.py
index 7bd94f8e490a..cc355ead1ff0 100755
--- a/tools/testing/selftests/tc-testing/tdc.py
+++ b/tools/testing/selftests/tc-testing/tdc.py
@@ -361,13 +361,22 @@ def run_one_test(pm, args, index, tidx):
     print("Test " + tidx["id"] + ": " + tidx["name"])
 
     if 'skip' in tidx:
+        if (args.verbose > 0):
+            print('probe command for test skip')
         if tidx['skip'] == 'yes':
-            res = TestResult(tidx['id'], tidx['name'])
-            res.set_result(ResultState.skip)
-            res.set_errormsg('Test case designated as skipped.')
-            pm.call_pre_case(tidx, test_skip=True)
-            pm.call_post_execute()
-            return res
+            # 'yes' would block forever: preserve existing skipped test
+            #  replacing 'yes' with 'false'
+            (p, procout) = exec_cmd(args, pm, 'execute', '/bin/false')
+        else:
+            (p, procout) = exec_cmd(args, pm, 'execute', tidx['skip'])
+        if p:
+            if (p.returncode != 0):
+                res = TestResult(tidx['id'], tidx['name'])
+                res.set_result(ResultState.skip)
+                res.set_errormsg('probe command failed: test skipped.')
+                pm.call_pre_case(tidx, test_skip=True)
+                pm.call_post_execute()
+                return res
 
     # populate NAMES with TESTID for this test
     NAMES['TESTID'] = tidx['id']
-- 
2.39.2