lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <Z9QtUoFdCoTXnkzE@pathway.suse.cz>
Date: Fri, 14 Mar 2025 14:21:22 +0100
From: Petr Mladek <pmladek@...e.com>
To: Tamir Duberstein <tamird@...il.com>
Cc: David Gow <davidgow@...gle.com>, Steven Rostedt <rostedt@...dmis.org>,
	Andy Shevchenko <andriy.shevchenko@...ux.intel.com>,
	Rasmus Villemoes <linux@...musvillemoes.dk>,
	Sergey Senozhatsky <senozhatsky@...omium.org>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Shuah Khan <shuah@...nel.org>,
	Geert Uytterhoeven <geert@...ux-m68k.org>,
	linux-kernel@...r.kernel.org, linux-kselftest@...r.kernel.org
Subject: Re: [PATCH v9 6/6] scanf: further break kunit into test cases

On Fri 2025-03-07 06:27:39, Tamir Duberstein wrote:
> This gives us more granular test output.
> 
> Output before:
> 
> > KTAP version 1
> > 1..1
> >     KTAP version 1
> >     # Subtest: scanf
> >     # module: scanf_kunit
> >     1..10
> >     ok 1 numbers_simple
> >         KTAP version 1
> >         # Subtest: numbers_list
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list: pass:5 fail:0 skip:0 total:5
> >     ok 2 numbers_list
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_typemax
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_typemax: pass:5 fail:0 skip:0 total:5
> >     ok 3 numbers_list_field_width_typemax
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_width
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_val_width: pass:5 fail:0 skip:0 total:5
> >     ok 4 numbers_list_field_width_val_width
> >     ok 5 numbers_slice
> >     ok 6 numbers_prefix_overflow
> >     ok 7 test_simple_strtoull
> >     ok 8 test_simple_strtoll
> >     ok 9 test_simple_strtoul
> >     ok 10 test_simple_strtol
> > # scanf: pass:10 fail:0 skip:0 total:10
> > # Totals: pass:22 fail:0 skip:0 total:22
> > ok 1 scanf
> 
> Output after:
> 
> > KTAP version 1
> > 1..1
> >     KTAP version 1
> >     # Subtest: scanf
> >     # module: scanf_kunit
> >     1..21
> >     ok 1 numbers_simple
> >         KTAP version 1
> >         # Subtest: numbers_list_ll
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_ll: pass:5 fail:0 skip:0 total:5
> >     ok 2 numbers_list_ll
> >         KTAP version 1
> >         # Subtest: numbers_list_l
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_l: pass:5 fail:0 skip:0 total:5
> >     ok 3 numbers_list_l
> >         KTAP version 1
> >         # Subtest: numbers_list_d
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_d: pass:5 fail:0 skip:0 total:5
> >     ok 4 numbers_list_d
> >         KTAP version 1
> >         # Subtest: numbers_list_h
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_h: pass:5 fail:0 skip:0 total:5
> >     ok 5 numbers_list_h
> >         KTAP version 1
> >         # Subtest: numbers_list_hh
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_hh: pass:5 fail:0 skip:0 total:5
> >     ok 6 numbers_list_hh
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_ll
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_ll: pass:5 fail:0 skip:0 total:5
> >     ok 7 numbers_list_field_width_ll
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_l
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_l: pass:5 fail:0 skip:0 total:5
> >     ok 8 numbers_list_field_width_l
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_d
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_d: pass:5 fail:0 skip:0 total:5
> >     ok 9 numbers_list_field_width_d
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_h
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_h: pass:5 fail:0 skip:0 total:5
> >     ok 10 numbers_list_field_width_h
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_hh
> >         ok 1 " "
> >         ok 2 ":"
> >         ok 3 ","
> >         ok 4 "-"
> >         ok 5 "/"
> >     # numbers_list_field_width_hh: pass:5 fail:0 skip:0 total:5
> >     ok 11 numbers_list_field_width_hh
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_ll
> >         ok 1 ""
> >         ok 2 " "
> >         ok 3 ":"
> >         ok 4 ","
> >         ok 5 "-"
> >         ok 6 "/"
> >     # numbers_list_field_width_val_ll: pass:6 fail:0 skip:0 total:6
> >     ok 12 numbers_list_field_width_val_ll
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_l
> >         ok 1 ""
> >         ok 2 " "
> >         ok 3 ":"
> >         ok 4 ","
> >         ok 5 "-"
> >         ok 6 "/"
> >     # numbers_list_field_width_val_l: pass:6 fail:0 skip:0 total:6
> >     ok 13 numbers_list_field_width_val_l
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_d
> >         ok 1 ""
> >         ok 2 " "
> >         ok 3 ":"
> >         ok 4 ","
> >         ok 5 "-"
> >         ok 6 "/"
> >     # numbers_list_field_width_val_d: pass:6 fail:0 skip:0 total:6
> >     ok 14 numbers_list_field_width_val_d
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_h
> >         ok 1 ""
> >         ok 2 " "
> >         ok 3 ":"
> >         ok 4 ","
> >         ok 5 "-"
> >         ok 6 "/"
> >     # numbers_list_field_width_val_h: pass:6 fail:0 skip:0 total:6
> >     ok 15 numbers_list_field_width_val_h
> >         KTAP version 1
> >         # Subtest: numbers_list_field_width_val_hh
> >         ok 1 ""
> >         ok 2 " "
> >         ok 3 ":"
> >         ok 4 ","
> >         ok 5 "-"
> >         ok 6 "/"
> >     # numbers_list_field_width_val_hh: pass:6 fail:0 skip:0 total:6
> >     ok 16 numbers_list_field_width_val_hh
> >     ok 17 numbers_prefix_overflow
> >     ok 18 test_simple_strtoull
> >     ok 19 test_simple_strtoll
> >     ok 20 test_simple_strtoul
> >     ok 21 test_simple_strtol
> > # scanf: pass:21 fail:0 skip:0 total:21
> > # Totals: pass:86 fail:0 skip:0 total:86
> > ok 1 scanf
> 
> Signed-off-by: Tamir Duberstein <tamird@...il.com>

Thanks for trying this. But I personally do not find this as a big win.
It causes that the test log is longer than one screen and thus harder
to review.

It might be fine when running only this test and running it only once.
But I think that it is quite common to run more tests and then
it is easy to get lost.

I guess that there are tools for handling the KTAP format which might
make this easier but...

So, I would prefer to avoid this patch.

Best Regards,
Petr

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ