netdev - Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAHS8izNyJgr=rFM8H_v=A3yfqdoDJysOtKA0CuZHRfgZ11ec4g@mail.gmail.com>
Date: Wed, 28 Aug 2024 11:26:28 -0700
From: Mina Almasry <almasrymina@...gle.com>
To: Willem de Bruijn <willemdebruijn.kernel@...il.com>
Cc: netdev@...r.kernel.org, davem@...emloft.net, kuba@...nel.org, 
	edumazet@...gle.com, pabeni@...hat.com, ncardwell@...gle.com, 
	shuah@...nel.org, linux-kselftest@...r.kernel.org, fw@...len.de, 
	Willem de Bruijn <willemb@...gle.com>
Subject: Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with ksft

On Tue, Aug 27, 2024 at 12:34 PM Willem de Bruijn
<willemdebruijn.kernel@...il.com> wrote:
>
> From: Willem de Bruijn <willemb@...gle.com>
>
> Lay the groundwork to import into kselftests the over 150 packetdrill
> TCP/IP conformance tests on github.com/google/packetdrill.
>
> Florian recently added support for packetdrill tests in nf_conntrack,
> in commit a8a388c2aae49 ("selftests: netfilter: add packetdrill based
> conntrack tests").
>
> This patch takes a slightly different implementation and reuses the
> ksft python library for its KTAP, ksft, NetNS and other such tooling.
>
> It also anticipates the large number of testcases, by creating a
> separate kselftest for each feature (directory). It does this by
> copying the template script packetdrill_ksft.py for each directory,
> and putting those in TEST_CUSTOM_PROGS so that kselftests runs each.
>
> To demonstrate the code with minimal patch size, initially import only
> two features/directories from github. One with a single script, and
> one with two. This was the only reason to pick tcp/inq and tcp/md5.
>
> Any future imports of packetdrill tests should require no additional
> coding. Just add the tcp/$FEATURE directory with *.pkt files.
>
> Implementation notes:
> - restore alphabetical order when adding the new directory to
>   tools/testing/selftests/Makefile
> - copied *.pkt files and support verbatim from the github project,
>   except for
>     - update common/defaults.sh path (there are two paths on github)
>     - add SPDX headers
>     - remove one author statement
>     - Acknowledgment: drop an e (checkpatch)
>
> Tested:
>         make -C tools/testing/selftests/ \
>           TARGETS=net/packetdrill \
>           install INSTALL_PATH=$KSFT_INSTALL_PATH
>
>         # in virtme-ng
>         sudo ./run_kselftest.sh -c net/packetdrill
>         sudo ./run_kselftest.sh -t net/packetdrill:tcp_inq.py

I did not know about run_kselftest.sh. From a quick look, it seems to
require selftests to be installed to run them. I think nipa relies on
something slightly different, it does something like this, AFAIU this
runs an individual kselftest without requiring it to be installed
first (which is nice):

make -C tools/testing/selftests TARGETS="net"
TEST_PROGS="rxtimestamp.sh" TEST_GEN_PROGS="" run_tests

It may be worth it to check if these added tests can be ran in a similar way.

This may also solve your issue with 'intermediate output' being
printed after every test. AFAIU if the tests can be ran individually
using a command like this, then you don't need intermediate output,
maybe.