linux-kernel - [PATCH 24/24] selftests/resctrl: Ignore failures from L2 CAT test with <= 2 bits

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20231024092634.7122-25-ilpo.jarvinen@linux.intel.com>
Date:   Tue, 24 Oct 2023 12:26:34 +0300
From:   Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>
To:     linux-kselftest@...r.kernel.org,
        Reinette Chatre <reinette.chatre@...el.com>,
        Shuah Khan <shuah@...nel.org>,
        Shaopeng Tan <tan.shaopeng@...fujitsu.com>,
        Maciej Wieczór-Retman 
        <maciej.wieczor-retman@...el.com>,
        Fenghua Yu <fenghua.yu@...el.com>
Cc:     linux-kernel@...r.kernel.org,
        Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>
Subject: [PATCH 24/24] selftests/resctrl: Ignore failures from L2 CAT test with <= 2 bits

L2 CAT test with low number of bits tends to occasionally fail because
of what seems random variation. The margin is quite small to begin with
for <= 2 bits in CBM. At times, the result can even become negative.
While it would be possible to allow negative values for those cases, it
would be more confusing to user.

Ignore failures from the tests where <= 2 were used to avoid false
negative results.

Signed-off-by: Ilpo Järvinen <ilpo.jarvinen@...ux.intel.com>
---
 tools/testing/selftests/resctrl/cat_test.c | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/resctrl/cat_test.c b/tools/testing/selftests/resctrl/cat_test.c
index a9c72022bb5a..bc88eb891f35 100644
--- a/tools/testing/selftests/resctrl/cat_test.c
+++ b/tools/testing/selftests/resctrl/cat_test.c
@@ -28,7 +28,7 @@
  */
 #define MIN_DIFF_PERCENT_PER_BIT	1
 
-static int show_results_info(__u64 sum_llc_val, int no_of_bits,
+static int show_results_info(__u64 sum_llc_val, int no_of_bits, bool ignore_fail,
 			     unsigned long cache_span, long min_diff_percent,
 			     unsigned long num_of_runs, bool platform,
 			     __s64 *prev_avg_llc_val)
@@ -40,12 +40,18 @@ static int show_results_info(__u64 sum_llc_val, int no_of_bits,
 	avg_llc_val = sum_llc_val / num_of_runs;
 	if (*prev_avg_llc_val) {
 		float delta = (__s64)(avg_llc_val - *prev_avg_llc_val);
+		char *res_str;
 
 		avg_diff = delta / *prev_avg_llc_val;
 		ret = platform && (avg_diff * 100) < (float)min_diff_percent;
 
+		res_str = ret ? "Fail:" : "Pass:";
+		if (ret && ignore_fail) {
+			res_str = "Pass (failure ignored):";
+			ret = 0;
+		}
 		ksft_print_msg("%s Check cache miss rate changed more than %.1f%%\n",
-			       ret ? "Fail:" : "Pass:", (float)min_diff_percent);
+			       res_str, (float)min_diff_percent);
 
 		ksft_print_msg("Percent diff=%.1f\n", avg_diff * 100);
 	}
@@ -85,6 +91,7 @@ static int check_results(struct resctrl_val_param *param, const char *cache_type
 
 	while (fgets(temp, sizeof(temp), fp)) {
 		char *token = strtok(temp, ":\t");
+		bool ignore_fail = false;
 		int fields = 0;
 		int bits;
 
@@ -108,7 +115,15 @@ static int check_results(struct resctrl_val_param *param, const char *cache_type
 
 		bits = count_bits(current_mask);
 
-		ret = show_results_info(sum_llc_perf_miss, bits,
+		/*
+		 * L2 CAT test with low number of bits has too small margin to
+		 * always remain positive. As negative values would be confusing
+		 * for the user, ignore failure instead.
+		 */
+		if (bits <= 2 && !strcmp(cache_type, "L2"))
+			ignore_fail = true;
+
+		ret = show_results_info(sum_llc_perf_miss, bits, ignore_fail,
 					alloc_size / 64,
 					MIN_DIFF_PERCENT_PER_BIT * (bits - 1), runs,
 					get_vendor() == ARCH_INTEL,
-- 
2.30.2