linux-kernel - Re: [PATCH 1/6] kunit: add example test case showing off all the expect macros

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CABVgOS=-KS8kdEsDFRuFM672KSvRGYEjtToA2J_-UeHLqcTH1Q@mail.gmail.com>
Date:   Tue, 11 Jan 2022 14:50:51 +0800
From:   David Gow <davidgow@...gle.com>
To:     Daniel Latypov <dlatypov@...gle.com>
Cc:     Brendan Higgins <brendanhiggins@...gle.com>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
        KUnit Development <kunit-dev@...glegroups.com>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Shuah Khan <skhan@...uxfoundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: Re: [PATCH 1/6] kunit: add example test case showing off all the
 expect macros

On Sat, Jan 8, 2022 at 9:23 AM Daniel Latypov <dlatypov@...gle.com> wrote:
>
> Currently, these macros are only really documented near the bottom of
> https://www.kernel.org/doc/html/latest/dev-tools/kunit/api/test.html#c.KUNIT_FAIL.
>
> E.g. it's likely someone might just not realize that
> KUNIT_EXPECT_STREQ() exists and instead use KUNIT_EXPECT_FALSE(strcmp())
> or similar.
>
> This can also serve as a basic smoketest that the KUnit assert machinery
> still works for all the macros.
>
> Signed-off-by: Daniel Latypov <dlatypov@...gle.com>
> ---

I think this is a great idea. I will note that this definitely isn't a
full test _of_ the assertion macros (in that it only exercises the
success case), so keeping it as an example is probably best.

A few possible ideas below, but I'm happy enough with this as-is regardless.

Reviewed-by: David Gow <davidgow@...gle.com>

>  lib/kunit/kunit-example-test.c | 46 ++++++++++++++++++++++++++++++++++
>  1 file changed, 46 insertions(+)
>
> diff --git a/lib/kunit/kunit-example-test.c b/lib/kunit/kunit-example-test.c
> index 51099b0ca29c..182a64c12541 100644
> --- a/lib/kunit/kunit-example-test.c
> +++ b/lib/kunit/kunit-example-test.c
> @@ -69,6 +69,51 @@ static void example_mark_skipped_test(struct kunit *test)
>         /* This line should run */
>         kunit_info(test, "You should see this line.");
>  }
> +
> +/*
> + * This test shows off all the KUNIT_EXPECT macros.
> + */
> +static void example_all_expect_macros_test(struct kunit *test)
> +{
> +       KUNIT_EXPECT_TRUE(test, true);
> +       KUNIT_EXPECT_FALSE(test, false);

_Maybe_ it's worth having a comment for each of these groups ('boolean
assertions', 'integer assertions', 'pointer assertions', etc)?

> +
> +       KUNIT_EXPECT_EQ(test, 1, 1);
> +       KUNIT_EXPECT_GE(test, 1, 1);
> +       KUNIT_EXPECT_LE(test, 1, 1);
> +       KUNIT_EXPECT_NE(test, 1, 0);
> +       KUNIT_EXPECT_GT(test, 1, 0);
> +       KUNIT_EXPECT_LT(test, 0, 1);
> +
> +       KUNIT_EXPECT_NOT_ERR_OR_NULL(test, test);
> +       KUNIT_EXPECT_PTR_EQ(test, NULL, NULL);
> +       KUNIT_EXPECT_PTR_NE(test, test, NULL);
> +
> +       KUNIT_EXPECT_STREQ(test, "hi", "hi");
> +       KUNIT_EXPECT_STRNEQ(test, "hi", "bye");
> +
> +       /*
> +        * There are also _MSG variants of all of the above that let you include
> +        * additional text on failure.
> +        */

There are also the ASSERT vs EXPECT variations. While it may be
excessive to also include all of these, particularly in an example, it
might be worth mentioning them in a comment somewhere?

Alternatively, if this is bloating the example too much, we could have
only one example each of the ASSERT and _MSG variants.

> +       KUNIT_EXPECT_TRUE_MSG(test, true, "msg");
> +       KUNIT_EXPECT_FALSE_MSG(test, false, "msg");

Part of me feels that a better message than "msg" would be nice to
have here, but I can't think of a good one. Maybe (particularly for
the less obvious integer/string/pointer macros below), having a
description of what's being asserted?



> +
> +       KUNIT_EXPECT_EQ_MSG(test, 1, 1, "msg");
> +       KUNIT_EXPECT_GE_MSG(test, 1, 1, "msg");
> +       KUNIT_EXPECT_LE_MSG(test, 1, 1, "msg");
> +       KUNIT_EXPECT_NE_MSG(test, 1, 0, "msg");
> +       KUNIT_EXPECT_GT_MSG(test, 1, 0, "msg");
> +       KUNIT_EXPECT_LT_MSG(test, 0, 1, "msg");
> +
> +       KUNIT_EXPECT_NOT_ERR_OR_NULL_MSG(test, test, "msg");
> +       KUNIT_EXPECT_PTR_EQ_MSG(test, NULL, NULL, "msg");
> +       KUNIT_EXPECT_PTR_NE_MSG(test, test, NULL, "msg");
> +
> +       KUNIT_EXPECT_STREQ_MSG(test, "hi", "hi", "msg");
> +       KUNIT_EXPECT_STRNEQ_MSG(test, "hi", "bye", "msg");
> +}
> +
>  /*
>   * Here we make a list of all the test cases we want to add to the test suite
>   * below.
> @@ -83,6 +128,7 @@ static struct kunit_case example_test_cases[] = {
>         KUNIT_CASE(example_simple_test),
>         KUNIT_CASE(example_skip_test),
>         KUNIT_CASE(example_mark_skipped_test),
> +       KUNIT_CASE(example_all_expect_macros_test),
>         {}
>  };
>
> --
> 2.34.1.575.g55b058a8bb-goog
>