netdev - Re: [PATCH wireguard] wireguard: selftests: refactor the test structure

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <YZ+C1MWdWQvd66ic@Laptop-X1>
Date:   Thu, 25 Nov 2021 20:34:28 +0800
From:   Hangbin Liu <liuhangbin@...il.com>
To:     "Jason A. Donenfeld" <Jason@...c4.com>
Cc:     Shuah Khan <shuah@...nel.org>,
        WireGuard mailing list <wireguard@...ts.zx2c4.com>,
        Netdev <netdev@...r.kernel.org>, linux-kselftest@...r.kernel.org
Subject: Re: [PATCH wireguard] wireguard: selftests: refactor the test
 structure

On Tue, Nov 16, 2021 at 03:35:40PM +0100, Jason A. Donenfeld wrote:
> Hi Hangbin,
> 
> I don't know how interested in this I am. Splitting things into two
> files means more confusing maintenance, and categorizing sections
> strictly into functions means there's more overhead when adding tests
> (e.g. "where do they fit?"), because the categories you've chosen are
> fairly broad, rather than being functions for each specific test. I'd
> be more amenable to something _entirely_ granular, because that'd be
> consistent, or what we have now, which is just linear. Full
> granularity, though, has its own downsides, of increased clutter.
> Alternatively, if you'd like to add some comments around the different
> areas to better document what's happening, perhaps that'd accomplish
> the same thing as this patch.
> 

Hi Jason,

May be my timezone is not very fit for yours. So I will copy my IRC replies
in the mail to moving on our kselftest topic.

The reason I did this patch is because I want to make the test more clear
and able to run each test case separately. My though is to make the
wireguard test looks like tools/testing/selftests/net/fib_tests.sh.(Of course
this could be discussed).

Because the linear structure makes reader hard to find out what test it does.
The function name in my current patch is also a little broad to look, which
could to be updated. After updating, I'd like to make the test has 2 parts,
functional tests and regression test. Functional tests for big part of function
tests and regression test for small specific issues.

BTW, one downside about current linear structure I think is that when someone
want to add a new test, he need to read through the whole test to know that
kind of topology at last. But with function structure, when we want to add a
new test. We can just do like:
1. set up basic topology
2. configure to specific topo for testing, or just skip the first step and
   configure to specific topo directly.
3. Do test
4. Clean up environment or reset to basic topology

I think this would make adding new test case easier. What do you think?

Thanks
Hangbin