linux-kernel - Re: [PATCH] lib/test_vmalloc.c: introduce xfail for failing tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <aGenwXvHCnOH2QBC@pc636>
Date: Fri, 4 Jul 2025 12:06:57 +0200
From: Uladzislau Rezki <urezki@...il.com>
To: Raghavendra K T <raghavendra.kt@....com>
Cc: akpm@...ux-foundation.org, urezki@...il.com,
	linux-kernel@...r.kernel.org, linux-mm@...ck.org
Subject: Re: [PATCH] lib/test_vmalloc.c: introduce xfail for failing tests

On Wed, Jul 02, 2025 at 06:43:19AM +0000, Raghavendra K T wrote:
> The test align_shift_alloc_test is expected to fail.
> Reporting the test as fail confuses to be a genuine failure.
> Introduce widely used xfail sematics to address the issue.
> 
> Note: a warn_alloc dump similar to below is still expected:
> 
>  Call Trace:
>   <TASK>
>   dump_stack_lvl+0x64/0x80
>   warn_alloc+0x137/0x1b0
>   ? __get_vm_area_node+0x134/0x140
> 
> Snippet of dmesg after change:
> 
> Summary: random_size_align_alloc_test passed: 1 failed: 0 xfailed: 0 ..
> Summary: align_shift_alloc_test passed: 0 failed: 0 xfailed: 1 ..
> Summary: pcpu_alloc_test passed: 1 failed: 0 xfailed: 0 ..
> 
> Signed-off-by: Raghavendra K T <raghavendra.kt@....com>
> ---
>  lib/test_vmalloc.c | 36 +++++++++++++++++++++---------------
>  1 file changed, 21 insertions(+), 15 deletions(-)
> 
> diff --git a/lib/test_vmalloc.c b/lib/test_vmalloc.c
> index 1b0b59549aaf..649f352e2046 100644
> --- a/lib/test_vmalloc.c
> +++ b/lib/test_vmalloc.c
> @@ -396,25 +396,27 @@ vm_map_ram_test(void)
>  struct test_case_desc {
>  	const char *test_name;
>  	int (*test_func)(void);
> +	bool xfail;
>  };
>  
>  static struct test_case_desc test_case_array[] = {
> -	{ "fix_size_alloc_test", fix_size_alloc_test },
> -	{ "full_fit_alloc_test", full_fit_alloc_test },
> -	{ "long_busy_list_alloc_test", long_busy_list_alloc_test },
> -	{ "random_size_alloc_test", random_size_alloc_test },
> -	{ "fix_align_alloc_test", fix_align_alloc_test },
> -	{ "random_size_align_alloc_test", random_size_align_alloc_test },
> -	{ "align_shift_alloc_test", align_shift_alloc_test },
> -	{ "pcpu_alloc_test", pcpu_alloc_test },
> -	{ "kvfree_rcu_1_arg_vmalloc_test", kvfree_rcu_1_arg_vmalloc_test },
> -	{ "kvfree_rcu_2_arg_vmalloc_test", kvfree_rcu_2_arg_vmalloc_test },
> -	{ "vm_map_ram_test", vm_map_ram_test },
> +	{ "fix_size_alloc_test", fix_size_alloc_test, },
> +	{ "full_fit_alloc_test", full_fit_alloc_test, },
> +	{ "long_busy_list_alloc_test", long_busy_list_alloc_test, },
> +	{ "random_size_alloc_test", random_size_alloc_test, },
> +	{ "fix_align_alloc_test", fix_align_alloc_test, },
> +	{ "random_size_align_alloc_test", random_size_align_alloc_test, },
> +	{ "align_shift_alloc_test", align_shift_alloc_test, true },
> +	{ "pcpu_alloc_test", pcpu_alloc_test, },
> +	{ "kvfree_rcu_1_arg_vmalloc_test", kvfree_rcu_1_arg_vmalloc_test, },
> +	{ "kvfree_rcu_2_arg_vmalloc_test", kvfree_rcu_2_arg_vmalloc_test, },
> +	{ "vm_map_ram_test", vm_map_ram_test, },
>  	/* Add a new test case here. */
>  };
>  
>  struct test_case_data {
>  	int test_failed;
> +	int test_xfailed;
>  	int test_passed;
>  	u64 time;
>  };
> @@ -444,7 +446,7 @@ static int test_func(void *private)
>  {
>  	struct test_driver *t = private;
>  	int random_array[ARRAY_SIZE(test_case_array)];
> -	int index, i, j;
> +	int index, i, j, ret;
>  	ktime_t kt;
>  	u64 delta;
>  
> @@ -468,11 +470,14 @@ static int test_func(void *private)
>  		 */
>  		if (!((run_test_mask & (1 << index)) >> index))
>  			continue;
> -
>  		kt = ktime_get();
>  		for (j = 0; j < test_repeat_count; j++) {
> -			if (!test_case_array[index].test_func())
> +			ret = test_case_array[index].test_func();
> +
> +			if (!ret && !test_case_array[index].xfail)
>  				t->data[index].test_passed++;
> +			else if (ret && test_case_array[index].xfail)
> +				t->data[index].test_xfailed++;
>  			else
>  				t->data[index].test_failed++;
>  		}
> @@ -576,10 +581,11 @@ static void do_concurrent_test(void)
>  				continue;
>  
>  			pr_info(
> -				"Summary: %s passed: %d failed: %d repeat: %d loops: %d avg: %llu usec\n",
> +				"Summary: %s passed: %d failed: %d xfailed: %d repeat: %d loops: %d avg: %llu usec\n",
>  				test_case_array[j].test_name,
>  				t->data[j].test_passed,
>  				t->data[j].test_failed,
> +				t->data[j].test_xfailed,
>  				test_repeat_count, test_loop_count,
>  				t->data[j].time);
>  		}
> -- 
> 2.43.0
> 

Reviewed-by: "Uladzislau Rezki (Sony)" <urezki@...il.com>

--
Uladzislau Rezki