Message-ID: <CABVgOSnTGFgzje3QZfN69We+26oWrWKoJQQP_=zphSLk4PG1xA@mail.gmail.com>
Date:   Thu, 14 Sep 2023 17:06:08 +0800
From:   David Gow <davidgow@...gle.com>
To:     Rae Moar <rmoar@...gle.com>
Cc:     shuah@...nel.org, dlatypov@...gle.com, brendan.higgins@...ux.dev,
        sadiyakazi@...gle.com, linux-kselftest@...r.kernel.org,
        kunit-dev@...glegroups.com, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2 1/2] kunit: add ability to run tests after boot using debugfs

On Sat, 9 Sept 2023 at 05:31, Rae Moar <rmoar@...gle.com> wrote:
>
> Add functionality to run built-in tests after boot by writing to a
> debugfs file.
>
> Add a new debugfs file labeled "run" for each test suite to use for
> this purpose.
>
> As an example, write to the file using the following:
>
> echo "any string" > /sys/kernel/debugfs/kunit/<testsuite>/run
>
> This will trigger the test suite to run and will print results to the
> kernel log.
>
> Note that what you "write" to the debugfs file will not be saved.
>
> To guard against running tests concurrently with this feature, add a
> mutex lock around running kunit. This supports the current practice of
> not allowing tests to be run concurrently on the same kernel.
>
> This functionality may not work for all tests.
>
> This new functionality could be used to design a parameter
> injection feature in the future.
>
> Signed-off-by: Rae Moar <rmoar@...gle.com>
> ---

This is looking pretty good, but I have a few nitpicks below and one big issue.

The big issue is that this doesn't seem to exclude test suites created
with kunit_test_init_section_suite{,s}(). The init section versions of
the suite declarations, by definition, won't work if run after the
kernel has finished booting. At the moment, these macros just pass
through to the normal versions (because we've not been able to run
after boot until now), but we'll need to implement it (maybe as a
separate linker section, maybe as an attribute, etc) now. I expect
that the correct solution here would be to not create the 'run'
debugfs file for these tests. But I could be convinced to have it
exist, but to just say "this test cannot be run after boot" if you've
got a good argument. In any case, grep 'test.h' for "NOTE TO KUNIT
DEVS" and you'll see the details.

My one other not-totally-related thought (and this extends to module
loading, too, so is possibly more useful as a separate patch) is that
we're continually incrementing the test number still. This doesn't
matter if we read the results from debugfs though, and it may make
more sense to have this continue to increment (and thus treat all of
dmesg as one long KTAP document). We could always add a reset option
to debugfs in a follow-up patch if we want. But that's not something
I'd hold this up with.

>
> Changes since v1:
> - Removed second patch as this problem has been fixed
> - Added Documentation patch
> - Made changes to work with new dynamically-extending log feature
>
> Note that these patches now rely on (and are rebased on) the patch series:
> https://lore.kernel.org/all/20230828104111.2394344-1-rf@opensource.cirrus.com/
>
>  lib/kunit/debugfs.c | 66 +++++++++++++++++++++++++++++++++++++++++++++
>  lib/kunit/test.c    | 13 +++++++++
>  2 files changed, 79 insertions(+)
>
> diff --git a/lib/kunit/debugfs.c b/lib/kunit/debugfs.c
> index 270d185737e6..8c0a970321ce 100644
> --- a/lib/kunit/debugfs.c
> +++ b/lib/kunit/debugfs.c
> @@ -8,12 +8,14 @@
>  #include <linux/module.h>
>
>  #include <kunit/test.h>
> +#include <kunit/test-bug.h>
>
>  #include "string-stream.h"
>  #include "debugfs.h"
>
>  #define KUNIT_DEBUGFS_ROOT             "kunit"
>  #define KUNIT_DEBUGFS_RESULTS          "results"
> +#define KUNIT_DEBUGFS_RUN              "run"
>
>  /*
>   * Create a debugfs representation of test suites:
> @@ -21,6 +23,8 @@
>   * Path                                                Semantics
>   * /sys/kernel/debug/kunit/<testsuite>/results Show results of last run for
>   *                                             testsuite
> + * /sys/kernel/debug/kunit/<testsuite>/run     Write to this file to trigger
> + *                                             testsuite to run
>   *
>   */
>
> @@ -99,6 +103,51 @@ static int debugfs_results_open(struct inode *inode, struct file *file)
>         return single_open(file, debugfs_print_results, suite);
>  }
>
> +/*
> + * Print a usage message to the debugfs "run" file
> + * (/sys/kernel/debug/kunit/<testsuite>/run) if opened.
> + */
> +static int debugfs_print_run(struct seq_file *seq, void *v)
> +{
> +       struct kunit_suite *suite = (struct kunit_suite *)seq->private;
> +
> +       seq_puts(seq, "Write to this file to trigger the test suite to run.\n");
> +       seq_printf(seq, "usage: echo \"any string\" > /sys/kernel/debugfs/kunit/%s/run\n",
> +                       suite->name);
> +       return 0;
> +}
> +
> +/*
> + * The debugfs "run" file (/sys/kernel/debug/kunit/<testsuite>/run)
> + * contains no information. Write to the file to trigger the test suite
> + * to run.
> + */
> +static int debugfs_run_open(struct inode *inode, struct file *file)
> +{
> +       struct kunit_suite *suite;
> +
> +       suite = (struct kunit_suite *)inode->i_private;
> +
> +       return single_open(file, debugfs_print_run, suite);
> +}
> +
> +/*
> + * Trigger a test suite to run by writing to the suite's "run" debugfs
> + * file found at: /sys/kernel/debug/kunit/<testsuite>/run
> + *
> + * Note: what is written to this file will not be saved.
> + */
> +static ssize_t debugfs_run(struct file *file,
> +               const char __user *buf, size_t count, loff_t *ppos)
> +{
> +       struct inode *f_inode = file->f_inode;
> +       struct kunit_suite *suite = (struct kunit_suite *) f_inode->i_private;
> +
> +       __kunit_test_suites_init(&suite, 1);
> +
> +       return count;
> +}
> +
>  static const struct file_operations debugfs_results_fops = {
>         .open = debugfs_results_open,
>         .read = seq_read,
> @@ -106,10 +155,23 @@ static const struct file_operations debugfs_results_fops = {
>         .release = debugfs_release,
>  };
>
> +static const struct file_operations debugfs_run_fops = {
> +       .open = debugfs_run_open,
> +       .read = seq_read,
> +       .write = debugfs_run,
> +       .llseek = seq_lseek,
> +       .release = debugfs_release,
> +};
> +
>  void kunit_debugfs_create_suite(struct kunit_suite *suite)
>  {
>         struct kunit_case *test_case;
>
> +       if (suite->log) {
> +               /* Clear the suite log that's leftover from a previous run. */
> +               string_stream_clear(suite->log);
> +               return;
> +       }

Can we just move this to kunit_init_suite() in test.c. It doesn't feel
quite debugfs-y enough, and the return here tripped me up for a little
too long.

Ideally, we'd then split up kunit_init_suite() into a one-time
initialisation (which calls kunit_debugfs_create_suite()), and a reset
function (which resets the state of the suite back to the beginning).
We then only call init once, but reset on every execution.
>         /* Allocate logs before creating debugfs representation. */
>         suite->log = alloc_string_stream(GFP_KERNEL);
>         string_stream_set_append_newlines(suite->log, true);
> @@ -124,6 +186,10 @@ void kunit_debugfs_create_suite(struct kunit_suite *suite)
>         debugfs_create_file(KUNIT_DEBUGFS_RESULTS, S_IFREG | 0444,
>                             suite->debugfs,
>                             suite, &debugfs_results_fops);
> +
> +       debugfs_create_file(KUNIT_DEBUGFS_RUN, S_IFREG | 0644,
> +                           suite->debugfs,
> +                           suite, &debugfs_run_fops);
>  }
>
>  void kunit_debugfs_destroy_suite(struct kunit_suite *suite)
> diff --git a/lib/kunit/test.c b/lib/kunit/test.c
> index 651cbda9f250..d376b886d72d 100644
> --- a/lib/kunit/test.c
> +++ b/lib/kunit/test.c
> @@ -13,6 +13,7 @@
>  #include <linux/kernel.h>
>  #include <linux/module.h>
>  #include <linux/moduleparam.h>
> +#include <linux/mutex.h>
>  #include <linux/panic.h>
>  #include <linux/sched/debug.h>
>  #include <linux/sched.h>
> @@ -22,6 +23,8 @@
>  #include "string-stream.h"
>  #include "try-catch-impl.h"
>
> +static struct mutex kunit_run_lock;
> +

Should we use DEFINE_MUTEX() here rather than initialising it at runtime?

>  /*
>   * Hook to fail the current test and print an error message to the log.
>   */
> @@ -668,6 +671,11 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_
>                 return 0;
>         }
>
> +       /* Use mutex lock to guard against running tests concurrently. */
> +       if (mutex_lock_interruptible(&kunit_run_lock)) {
> +               pr_err("kunit: test interrupted\n");
> +               return -EINTR;
> +       }
>         static_branch_inc(&kunit_running);
>
>         for (i = 0; i < num_suites; i++) {
> @@ -676,6 +684,7 @@ int __kunit_test_suites_init(struct kunit_suite * const * const suites, int num_
>         }
>
>         static_branch_dec(&kunit_running);
> +       mutex_unlock(&kunit_run_lock);
>         return 0;
>  }
>  EXPORT_SYMBOL_GPL(__kunit_test_suites_init);
> @@ -836,6 +845,10 @@ static int __init kunit_init(void)
>         kunit_install_hooks();
>
>         kunit_debugfs_init();
> +
> +       /* Initialize lock to guard against running tests concurrently. */
> +       mutex_init(&kunit_run_lock);
> +

As I understand it, we can just use DEFINE_MUTEX() above.


>  #ifdef CONFIG_MODULES
>         return register_module_notifier(&kunit_mod_nb);
>  #else
>
> base-commit: b754593274e04fc840482a658b29791bc8f8b933
> --
> 2.42.0.283.g2d96d420d3-goog
>

Download attachment "smime.p7s" of type "application/pkcs7-signature" (4003 bytes)

Powered by blists - more mailing lists