linux-kernel - Re: Issue in parsing of tests that use KUNIT_CASE

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <aUOrH1iapKnOaZEj@li-dc0c254c-257c-11b2-a85c-98b6c1322444.ibm.com>
Date: Thu, 18 Dec 2025 12:49:59 +0530
From: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
To: linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
        Shuah Khan <skhan@...uxfoundation.org>,
        Brendan Higgins <brendan.higgins@...ux.dev>,
        David Gow <davidgow@...gle.com>, Rae Moar <raemoar63@...il.com>
Cc: linux-kernel@...r.kernel.org
Subject: Re: Issue in parsing of tests that use KUNIT_CASE_PARAM

On Wed, Dec 17, 2025 at 08:47:57PM +0530, Ojaswin Mujoo wrote:
> Hello,
> 
> While writing some Kunit tests for ext4 filesystem, I'm encountering an
> issue in the way we display the diagnostic logs upon failures, when
> using KUNIT_CASE_PARAM() to write the tests.
> 
> This can be observed by patching fs/ext4/mballoc-test.c to fail
> and print one of the params:
> 
> --- a/fs/ext4/mballoc-test.c
> +++ b/fs/ext4/mballoc-test.c
> @@ -350,6 +350,8 @@ static int mbt_kunit_init(struct kunit *test)
>         struct super_block *sb;
>         int ret;
> 
> +       KUNIT_FAIL(test, "Failed: blocksize_bits=%d", layout->blocksize_bits);
> +
>         sb = mbt_ext4_alloc_super_block();
>         if (sb == NULL)
>                 return -ENOMEM;
> 
> With the above change, we can observe the following output (snipped):
> 
> [18:50:25] ============== ext4_mballoc_test (7 subtests) ==============
> [18:50:25] ================= test_new_blocks_simple  ==================
> [18:50:25] [FAILED] block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
> [18:50:25]     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> [18:50:25] Failed: blocksize_bits=12
> [18:50:25] [FAILED] block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
> [18:50:25]     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> [18:50:25] Failed: blocksize_bits=16
> [18:50:25] [FAILED] block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
> [18:50:25]     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> [18:50:25] Failed: blocksize_bits=10
> [18:50:25]     # test_new_blocks_simple: pass:0 fail:3 skip:0 total:3
> [18:50:25] ============= [FAILED] test_new_blocks_simple ==============
> <snip>
> 
> Note that the diagnostic logs don't show up correctly. Ideally they
> should be before test result but here the first [FAILED] test has no
> logs printed above whereas the last "Failed: blocksize_bits=10" print
> comes after the last subtest, when it actually corresponds to the first
> subtest.
> 
> The KTAP file itself seems to have diagnostic logs in the right place:
> 
> KTAP version 1
> 1..2
>     KTAP version 1
>     # Subtest: ext4_mballoc_test
>     # module: ext4
>     1..7
>         KTAP version 1
>         # Subtest: test_new_blocks_simple

So looking into this a bit more and comparing the parameterized output
with non parameterized output, I'm seeing that the difference is that
output via KUNIT_CASE_PARAM is not printing the test plan line right
here. This plan sort of serves as divider between the parent and the 3
children's logs and without it our parsing logic gets confused. When I
manually added a "1..3" test plan I could see the parsing work correctly
without any changes to kunit_parser.py.

Regards,
ojaswin

>     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> Failed: blocksize_bits=10
>         not ok 1 block_bits=10 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
>     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> Failed: blocksize_bits=12
>         not ok 2 block_bits=12 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
>     # test_new_blocks_simple: EXPECTATION FAILED at fs/ext4/mballoc-test.c:364
> Failed: blocksize_bits=16
>         not ok 3 block_bits=16 cluster_bits=3 blocks_per_group=8192 group_count=4 desc_size=64
>     # test_new_blocks_simple: pass:0 fail:3 skip:0 total:3
>     not ok 1 test_new_blocks_simple
>     <snip>
> 
> By tracing kunit_parser.py script, I could see the issue here is in the
> parsing of the "Subtest: test_new_blocks_simple". We end up associating
> everything below the subtest till "not ok 1 block_bits=10..." as
> diagnostic logs of the subtest, while these lons actually belong to the
> first of the 3 subtests under this test.
>