netdev - Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <66d23da8e7cb7_3d822329461@willemb.c.googlers.com.notmuch>
Date: Fri, 30 Aug 2024 17:46:16 -0400
From: Willem de Bruijn <willemdebruijn.kernel@...il.com>
To: Willem de Bruijn <willemdebruijn.kernel@...il.com>, 
 Jakub Kicinski <kuba@...nel.org>, 
 Willem de Bruijn <willemdebruijn.kernel@...il.com>
Cc: Paolo Abeni <pabeni@...hat.com>, 
 netdev@...r.kernel.org, 
 davem@...emloft.net, 
 edumazet@...gle.com, 
 ncardwell@...gle.com, 
 shuah@...nel.org, 
 linux-kselftest@...r.kernel.org, 
 fw@...len.de, 
 Willem de Bruijn <willemb@...gle.com>, 
 "Matthieu Baerts (NGI0)" <matttbe@...nel.org>, 
 martineau@...nel.org
Subject: Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with
 ksft

Willem de Bruijn wrote:
> Jakub Kicinski wrote:
> > On Fri, 30 Aug 2024 11:20:05 -0400 Willem de Bruijn wrote:
> > > Kselftest install does not preserve directories.
> > > 
> > > So all .pkt files are copied into net/packetdrill root. This is messy.
> > > More fundamentally it breaks the includes in the files (e..g, `source
> > > ../common/defaults.sh`).
> > 
> > Can you show an example of exact commands and what happens?
> 
> Running directly works fine:
> 
>     $ KSELFTEST_PKT_INTERP=packetdrill_ksft.sh
>     $ make -C tools/testing/selftests \
>             TARGETS=net/packetdrill O=/tmp run_tests
> 
>     TAP version 13
>     1..3
>     # timeout set to 45
>     # selftests: net/packetdrill: client.pkt
>     # TAP version 13
>     # 1..2
>     # ok 1 ipv4
>     # ok 2 ipv6
>     # # Totals: pass:2 fail:0 xfail:0 xpass:0 skip:0 error:0
>     ok 1 selftests: net/packetdrill: client.pkt
>     [..etc..]
> 
> Installing does not:
> 
>     $ make -C tools/testing/selftests/ \
>           TARGETS=net/packetdrill \
>           install INSTALL_PATH=$INSTALL_DIR
>     $ cd $INSTALL_DIR
>     $ export KSELFTEST_PKT_INTERP=packetdrill_ksft.sh
>     $ ./run_kselftest.sh -c net/packetdrill
> 
>     TAP version 13
>     1..3
>     # timeout set to 45
>     # selftests: net/packetdrill: client.pkt
>     # TAP version 13
>     # 1..2
>     # sh: line 1: ../common/defaults.sh: No such file or directory
>     # ./client.pkt: error executing init command: non-zero status 127
>     # not ok 1 ipv4
>     # sh: line 1: ../common/defaults.sh: No such file or directory
>     # ./client.pkt: error executing init command: non-zero status 127
>     # not ok 2 ipv6
>     # # Totals: pass:0 fail:2 xfail:0 xpass:0 skip:0 error:0
>     not ok 1 selftests: net/packetdrill: client.pkt # exit=1
> 
> Due to that relative path to defaults.sh inside the scripts.
> 
> It is arguably a bit weird that the relative path of the TEST_PROGS
> differs before and after install.

I guess what's weird is more what I'm trying to do. There has been
no need so far for progs in subdirectories from the TARGET.

One option is to just create a bunch of targets with a wildcard.
Something like below (entirely untested).

TARGETS += net/packetdrill/tcp/$(wildcard *)

> > We have directories in net/lib, and it's a target, and it works, no?
> 
> net/lib is not a TARGET in tools/testing/selftests/Makefile. Its
> Makefile only generates dependencies for other targets: TEST_FILES,
> TEST_GEN_FILES and TEST_INCLUDES.
> 
> This issue with preserving paths until recently also existed for
> helper files (TEST_FILES). TEST_INCLUDES was added expressly to
> preserve those paths (commit 2a0683be5b4c).

Another option is to dive yet deeper into kselftest infra and do the
same for TEST_PROGS, either TEST_PROGS itself or as a new
TEST_PROGS_FULLPATH or so.

It needs a change to INSTALL_RULE (maybe just passing -R to rsync),
to emit_tests to skip basename when writing kselftest-list.txt, and
probably to run_one() to break up the path and chdir. I'll see if I
can make this work without too much churn.