linux-kernel - [PATCH 0/3] Introduce KUNIT_EXPECT_ARREQ and KUNIT_EXPECT

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Message-Id: <20220802161206.228707-1-mairacanal@riseup.net>
Date:   Tue,  2 Aug 2022 13:12:03 -0300
From:   Maíra Canal <mairacanal@...eup.net>
To:     Brendan Higgins <brendanhiggins@...gle.com>, davidgow@...gle.com,
        airlied@...ux.ie, daniel@...ll.ch, davem@...emloft.net,
        kuba@...nel.org, jose.exposito89@...il.com, javierm@...hat.com
Cc:     andrealmeid@...eup.net, melissa.srw@...il.com,
        siqueirajordao@...eup.net, Isabella Basso <isabbasso@...eup.net>,
        magalilemes00@...il.com, tales.aparecida@...il.com,
        linux-kselftest@...r.kernel.org, kunit-dev@...glegroups.com,
        linux-kernel@...r.kernel.org,
        Maíra Canal <mairacanal@...eup.net>
Subject: [PATCH 0/3] Introduce KUNIT_EXPECT_ARREQ and KUNIT_EXPECT_ARRNEQ macros

Currently, in order to compare arrays in KUnit, the KUNIT_EXPECT_EQ or
KUNIT_EXPECT_FALSE macros are used in conjunction with the memcmp function,
such as:
  KUNIT_EXPECT_EQ(test, memcmp(foo, bar, size), 0);

Although this usage produces correct results for the test cases, if the
expectation fails the error message is not very helpful, indicating only the
return of the memcmp function.

Therefore, create a new set of macros KUNIT_EXPECT_ARREQ and
KUNIT_EXPECT_ARRNEQ that compare memory blocks until a determined size. In
case of expectation failure, those macros print the hex dump of the memory
blocks, making it easier to debug test failures for arrays.

For example, if I am using the KUNIT_EXPECT_ARREQ macro and apply the
following diff (introducing a test failure) to the 
drm/tests/drm_format_helper.c:

diff --git a/drivers/gpu/drm/tests/drm_format_helper_test.c b/drivers/gpu/drm/tests/drm_format_helper_test.c
index 3106abb3bead..942aa131a768 100644
--- a/drivers/gpu/drm/tests/drm_format_helper_test.c
+++ b/drivers/gpu/drm/tests/drm_format_helper_test.c
@@ -131,9 +131,9 @@ static struct convert_xrgb8888_case convert_xrgb8888_cases[] = {
                    .rgb565_result = {
                        .dst_pitch = 10,
                        .expected = {
-                               0x0A33, 0x1260, 0xA800, 0x0000, 0x0000,
-                               0x6B8E, 0x0A33, 0x1260, 0x0000, 0x0000,
-                               0xA800, 0x6B8E, 0x0A33, 0x0000, 0x0000,
+                               0x0A31, 0x1260, 0xA800, 0x0000, 0x0000,
+                               0x6B81, 0x0A33, 0x1260, 0x0000, 0x0000,
+                               0xA801, 0x6B8E, 0x0A33, 0x0000, 0x0000,
                        },
                        .expected_swab = {
                                0x330A, 0x6012, 0x00A8, 0x0000, 0x0000,}}}

I will get a test failure with the following form:

➜ ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests \
  --kconfig_add CONFIG_UML_PCI_OVER_VIRTIO=y --kconfig_add CONFIG_VIRTIO_UML=y \
  'drm_format_helper_test'
  [...]
  [12:38:20] ================= xrgb8888_to_rgb565_test ==================
  [12:38:20] [PASSED] single_pixel_source_buffer
  [12:38:20] [PASSED] single_pixel_clip_rectangle
  [12:38:20] [PASSED] well_known_colors
  [12:38:20] # xrgb8888_to_rgb565_test: EXPECTATION FAILED at drivers/gpu/drm/tests/drm_format_helper_test.c:248
  [12:38:20] Expected dst == result->expected, but
  [12:38:20] dst ==
  [12:38:20] 00000000: 33 0a 60 12 00 a8 00 00 00 00 8e 6b 33 0a 60 12
  [12:38:20] 00000010: 00 00 00 00 00 a8 8e 6b 33 0a 00 00 00 00
  [12:38:20] result->expected ==
  [12:38:20] 00000000: 31 0a 60 12 00 a8 00 00 00 00 81 6b 33 0a 60 12
  [12:38:20] 00000010: 00 00 00 00 01 a8 8e 6b 33 0a 00 00 00 00
  [12:38:20] not ok 4 - destination_pitch
  [12:38:20] [FAILED] destination_pitch
  [12:38:20] # Subtest: xrgb8888_to_rgb565_test
  [12:38:20] # xrgb8888_to_rgb565_test: pass:3 fail:1 skip:0 total:4
  [12:38:20] not ok 2 - xrgb8888_to_rgb565_test
  [...]
  [12:38:20] ============= [FAILED] drm_format_helper_test ==============
  [12:38:20] ============================================================
  [12:38:20] Testing complete. Ran 8 tests: passed: 7, failed: 1
  [12:38:20] Elapsed time: 3.713s total, 0.002s configuring, 3.546s building, 0.135s running

Noticed that, with the hex dump, it is possible to check which bytes are
making the test fail. So, it is easier to debug the cause of the failure.

The first patch of the series introduces the KUNIT_EXPECT_ARREQ and
KUNIT_EXPECT_ARRNEQ. The second patch adds an example of array expectations
on the kunit-example-test.c. And the last patch replaces the KUNIT_EXPECT_EQ
for KUNIT_EXPECT_ARREQ on the existing occurrences.

Best Regards,
- Maíra Canal

Maíra Canal (3):
  kunit: Introduce KUNIT_EXPECT_ARREQ and KUNIT_EXPECT_ARRNEQ macros
  kunit: add KUnit array assertions to the example_all_expect_macros_test
  kunit: use KUNIT_EXPECT_ARREQ macro

 .../gpu/drm/tests/drm_format_helper_test.c    |  6 +-
 include/kunit/assert.h                        | 35 +++++++++
 include/kunit/test.h                          | 76 +++++++++++++++++++
 lib/kunit/assert.c                            | 43 +++++++++++
 lib/kunit/kunit-example-test.c                |  7 ++
 net/core/dev_addr_lists_test.c                |  4 +-
 6 files changed, 166 insertions(+), 5 deletions(-)

-- 
2.37.1