lists.openwall.net   lists  /  announce  owl-users  owl-dev  john-users  john-dev  passwdqc-users  yescrypt  popa3d-users  /  oss-security  kernel-hardening  musl  sabotage  tlsify  passwords  /  crypt-dev  xvendor  /  Bugtraq  Full-Disclosure  linux-kernel  linux-netdev  linux-ext4  linux-hardening  linux-cve-announce  PHC 
Open Source and information security mailing list archives
 
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20240828140035.4554142f@kernel.org>
Date: Wed, 28 Aug 2024 14:00:35 -0700
From: Jakub Kicinski <kuba@...nel.org>
To: Willem de Bruijn <willemdebruijn.kernel@...il.com>
Cc: Paolo Abeni <pabeni@...hat.com>, netdev@...r.kernel.org,
 davem@...emloft.net, edumazet@...gle.com, ncardwell@...gle.com,
 shuah@...nel.org, linux-kselftest@...r.kernel.org, fw@...len.de, Willem de
 Bruijn <willemb@...gle.com>, "Matthieu Baerts (NGI0)" <matttbe@...nel.org>,
 martineau@...nel.org
Subject: Re: [PATCH net-next RFC] selftests/net: integrate packetdrill with
 ksft

On Wed, 28 Aug 2024 15:33:33 -0400 Willem de Bruijn wrote:
> That could work.
> 
> Is reporting one KTAP and exitcode per directory vs per packetdrill
> invocation good/bad/neither?

To me "neither", hopefully and unhelpfully I won't tell you anything
that's not common sense :) The balance is between:

 - having test cases which don't take too long (assuming debug kernel):
     <15min is good, 
     >1h I will start complaining,
     >1h30m is bad because we can't retry and still make the 3h
            deadline that NIPA has,
     >3h the test can't run in NIPA at all.

vs

 - flip side is having so many cases it's hard to keep track and render
   in the UI. JSON is relatively slow to process. If you have 150 cases,
   that's 300 per branch (debug and non-debug kernels), times 8
   branches a day => 2.4k results / day
   I think that's still fine-ish, but on the larger side for sure. For
   reference net and forwarding have ~100 tests each. FWIW we do have
   the ability to collect and display nested KTAP so the information is
   not lost (but it makes queries slower so we don't fetch it by default).

> Three other issues if this is calling packetdrill directly is
> - passing the non-trivial IP specific flags
> - running twice, for IPv4 and IPv6
> - chdir into the directory of the pkt file
> 
> That can be addressed by instead calling a small wrapper shell script.
> 
> That would do the test_func_builder part of packetdrill_ksft.py.
> But without the need to handle netns, popen/cmd, etc, and thus the
> ksft dependencies.

Right!

Powered by blists - more mailing lists

Powered by Openwall GNU/*/Linux Powered by OpenVZ