linux-kernel - Re: [PATCH v3 0/9] kunit: Add test attributes API

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <e87a70f7-9b7e-e5dc-bc00-8f8ba76f0476@riseup.net>
Date:   Wed, 26 Jul 2023 11:56:57 -0300
From:   Arthur Grillo Queiroz Cabral <arthurgrillo@...eup.net>
To:     Rae Moar <rmoar@...gle.com>, shuah@...nel.org, davidgow@...gle.com,
        dlatypov@...gle.com, brendan.higgins@...ux.dev
Cc:     linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-kernel@...r.kernel.org, keescook@...omium.org,
        linux-hardening@...r.kernel.org, jstultz@...gle.com,
        tglx@...utronix.de, sboyd@...nel.org
Subject: Re: [PATCH v3 0/9] kunit: Add test attributes API



On 25/07/23 18:25, 'Rae Moar' via KUnit Development wrote:
> Hello everyone,
> 
> This patch series adds a test attributes framework to KUnit.
> 
> There has been interest in filtering out "slow" KUnit tests. Most notably,
> a new config, CONFIG_MEMCPY_SLOW_KUNIT_TEST, has been added to exclude a
> particularly slow memcpy test
> (https://lore.kernel.org/all/20230118200653.give.574-kees@kernel.org/).
> 
> This attributes framework can be used to save and access test associated
> data, including whether a test is slow. These attributes are reportable
> (via KTAP and command line output) and are also filterable.
> 
> This framework is designed to allow for the addition of other attributes in
> the future. These attributes could include whether the test can be run
> concurrently, test file path, etc.
> 
> To try out the framework I suggest running:
> "./tools/testing/kunit/kunit.py run --filter speed!=slow"
> 
> This patch series was originally sent out as an RFC. Here is a link to the
> RFC v2:
> https://lore.kernel.org/all/20230707210947.1208717-1-rmoar@google.com/
> 
> Thanks!
> Rae
> 

Great addition to the KUnit framework!

On all the patch set

Reviewed-by: Arthur Grillo <arthurgrillo@...eup.net>

Best Regards,
~Arthur Grillo

> Rae Moar (9):
>   kunit: Add test attributes API structure
>   kunit: Add speed attribute
>   kunit: Add module attribute
>   kunit: Add ability to filter attributes
>   kunit: tool: Add command line interface to filter and report
>     attributes
>   kunit: memcpy: Mark tests as slow using test attributes
>   kunit: time: Mark test as slow using test attributes
>   kunit: add tests for filtering attributes
>   kunit: Add documentation of KUnit test attributes
> 
>  Documentation/dev-tools/kunit/run_wrapper.rst |  12 +
>  .../dev-tools/kunit/running_tips.rst          | 166 +++++++
>  include/kunit/attributes.h                    |  50 +++
>  include/kunit/test.h                          |  70 ++-
>  kernel/time/time_test.c                       |   2 +-
>  lib/Kconfig.debug                             |   3 +
>  lib/kunit/Makefile                            |   3 +-
>  lib/kunit/attributes.c                        | 418 ++++++++++++++++++
>  lib/kunit/executor.c                          | 114 ++++-
>  lib/kunit/executor_test.c                     | 128 +++++-
>  lib/kunit/kunit-example-test.c                |   9 +
>  lib/kunit/test.c                              |  27 +-
>  lib/memcpy_kunit.c                            |   8 +-
>  tools/testing/kunit/kunit.py                  |  70 ++-
>  tools/testing/kunit/kunit_kernel.py           |   8 +-
>  tools/testing/kunit/kunit_parser.py           |  11 +-
>  tools/testing/kunit/kunit_tool_test.py        |  39 +-
>  17 files changed, 1062 insertions(+), 76 deletions(-)
>  create mode 100644 include/kunit/attributes.h
>  create mode 100644 lib/kunit/attributes.c
> 
> 
> base-commit: 64bd4641310c41a1ecf07c13c67bc0ed61045dfd