linux-kernel - [PATCH v2] kunit: tool: add ability to parse multiple files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240306223056.226518-1-rmoar@google.com>
Date: Wed,  6 Mar 2024 22:30:55 +0000
From: Rae Moar <rmoar@...gle.com>
To: shuah@...nel.org, davidgow@...gle.com, dlatypov@...gle.com, 
	brendan.higgins@...ux.dev, kevko@...gle.com
Cc: linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com, 
	linux-kernel@...r.kernel.org, Rae Moar <rmoar@...gle.com>
Subject: [PATCH v2] kunit: tool: add ability to parse multiple files

Add ability to parse multiple files. Additionally add the
ability to parse all results in the KUnit debugfs repository.

How to parse multiple files:

/tools/testing/kunit/kunit.py parse results.log results2.log

How to parse all files in directory:

/tools/testing/kunit/kunit.py parse directory_path/*

How to parse KUnit debugfs repository:

/tools/testing/kunit/kunit.py parse debugfs

For each file, the parser outputs the file name, results, and test
summary. At the end of all parsing, the parser outputs a total summary
line.

This feature can be easily tested on the tools/testing/kunit/test_data/
directory.

Signed-off-by: Rae Moar <rmoar@...gle.com>
---
Changes since v1:
- Annotate type of parsed_files
- Add ability to input file name from stdin again
- Make for loops a bit terser
- Add no output warning
- Change feature to take in multiple fields rather than a directory.
  Currently nonrecursive. Let me know if people would prefer this as
  recursive.

 tools/testing/kunit/kunit.py | 45 +++++++++++++++++++++++++-----------
 1 file changed, 32 insertions(+), 13 deletions(-)

diff --git a/tools/testing/kunit/kunit.py b/tools/testing/kunit/kunit.py
index bc74088c458a..df804a118aa5 100755
--- a/tools/testing/kunit/kunit.py
+++ b/tools/testing/kunit/kunit.py
@@ -511,19 +511,37 @@ def exec_handler(cli_args: argparse.Namespace) -> None:
 
 
 def parse_handler(cli_args: argparse.Namespace) -> None:
-	if cli_args.file is None:
+	parsed_files = [] # type: List[str]
+	total_test = kunit_parser.Test()
+	total_test.status = kunit_parser.TestStatus.SUCCESS
+	if cli_args.files is None:
 		sys.stdin.reconfigure(errors='backslashreplace')  # type: ignore
-		kunit_output = sys.stdin  # type: Iterable[str]
+		parsed_files.append(sys.stdin)
+	elif cli_args.files[0] == "debugfs" and len(cli_args.files) == 1:
+		for (root, _, files) in os.walk("/sys/kernel/debug/kunit"):
+			parsed_files.extend(os.path.join(root, f) for f in files if f == "results")
 	else:
-		with open(cli_args.file, 'r', errors='backslashreplace') as f:
+		parsed_files.extend(f for f in cli_args.files if os.path.isfile(f))
+
+	if len(parsed_files) == 0:
+		print("No output found.")
+
+	for file in parsed_files:
+		print(file)
+		with open(file, 'r', errors='backslashreplace') as f:
 			kunit_output = f.read().splitlines()
-	# We know nothing about how the result was created!
-	metadata = kunit_json.Metadata()
-	request = KunitParseRequest(raw_output=cli_args.raw_output,
-					json=cli_args.json)
-	result, _ = parse_tests(request, metadata, kunit_output)
-	if result.status != KunitStatus.SUCCESS:
-		sys.exit(1)
+		# We know nothing about how the result was created!
+		metadata = kunit_json.Metadata()
+		request = KunitParseRequest(raw_output=cli_args.raw_output,
+						json=cli_args.json)
+		_, test = parse_tests(request, metadata, kunit_output)
+		total_test.subtests.append(test)
+
+	if len(parsed_files) > 1: # if more than one file was parsed output total summary
+		print('All files parsed.')
+		stdout.print_with_timestamp(kunit_parser.DIVIDER)
+		kunit_parser.bubble_up_test_results(total_test)
+		kunit_parser.print_summary_line(total_test)
 
 
 subcommand_handlers_map = {
@@ -569,9 +587,10 @@ def main(argv: Sequence[str]) -> None:
 					    help='Parses KUnit results from a file, '
 					    'and parses formatted results.')
 	add_parse_opts(parse_parser)
-	parse_parser.add_argument('file',
-				  help='Specifies the file to read results from.',
-				  type=str, nargs='?', metavar='input_file')
+	parse_parser.add_argument('files',
+				  help='List of file paths to read results from or keyword'
+				  		'"debugfs" to read all results from the debugfs directory.',
+				  type=str, nargs='*', metavar='input_files')
 
 	cli_args = parser.parse_args(massage_argv(argv))
 

base-commit: 806cb2270237ce2ec672a407d66cee17a07d3aa2
-- 
2.44.0.278.ge034bb2e1d-goog