netdev - Re: [PATCH RFC net-next v4 2/2] selftests/dpll: add DPLL system integration selftests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20231201115259.37821ed5@kernel.org>
Date: Fri, 1 Dec 2023 11:52:59 -0800
From: Jakub Kicinski <kuba@...nel.org>
To: "Michalik, Michal" <michal.michalik@...el.com>
Cc: "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
 "vadim.fedorenko@...ux.dev" <vadim.fedorenko@...ux.dev>, "Kubalewski,
 Arkadiusz" <arkadiusz.kubalewski@...el.com>, "jonathan.lemon@...il.com"
 <jonathan.lemon@...il.com>, "pabeni@...hat.com" <pabeni@...hat.com>, poros
 <poros@...hat.com>, "Olech, Milena" <milena.olech@...el.com>, mschmidt
 <mschmidt@...hat.com>, "linux-clk@...r.kernel.org"
 <linux-clk@...r.kernel.org>, "bvanassche@....org" <bvanassche@....org>,
 "davem@...emloft.net" <davem@...emloft.net>, "edumazet@...gle.com"
 <edumazet@...gle.com>
Subject: Re: [PATCH RFC net-next v4 2/2] selftests/dpll: add DPLL system
 integration selftests

On Fri, 1 Dec 2023 18:33:11 +0000 Michalik, Michal wrote:
> That looks really promising - great idea. I tried only vmtest today, and my tests
> work kind of flawless with my own built kernel (nested VMs):
>   $ vmtest -k /home/net-next/vmlinux "modprobe netdevsim && KSRC=/home/net-next/ pytest"
>   => vmlinux
>   ===> Booting
>   ===> Setting up VM
>   ===> Running command  
>   ============================= test session starts ==============================
>   platform linux -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0
>   rootdir: /home/net-next/tools/testing/selftests/drivers/net/netdevsim/dpll
>   collected 91 items  
> 
>   test_dpll.py ........................................................... [ 64%]
>   ................................                                         [100%]
> 
>   ============================= 91 passed in 10.54s ==============================
> 
> I will try to take a look at virtme-ng next week, but to be frank I already like
> the vmtest.

Hm, FWIW I manged to get virtme-ng to work (I was pointing it at a
vmlinux not bzImage which it expects). But vmtest is still unhappy.

$ vmtest -k build/vmlinux "echo Running!"
=> vmlinux
===> Booting
Failed to connect QGA

Caused by:
    Timed out waiting for QGA connection


Are you on Ubuntu? I'm on Fedora. Maybe it has some distro deps :(

> >> To be honest I did not see that. I agree that this is a simpler solution, but I am
> >> not sure if that is not "too simple". What I mean, I'm not sure who wrote the sdsi
> >> tests, but maybe they were not aware about the Python best practices? Python used
> >> to be my first language, and I would vote for using the venvs if you asked me.
> >> I understand that it haven't been done before, but we are here to try to improve
> >> the things, yes?   
> > 
> > I think I already asked how long the setup takes but my only concern 
> > is that the setup will be slower, and less useful during development.
> 
> I wanted for "run_dpll_test.sh" to be userfriendly even for people who does not
> have a clue how python/pytest works. If somebody is developing tests, I assume
> he/she knows what she is doing and is using own environment either way, like
> venvs with additional Python debug tools and direct pytest in tests directory:
>   KSRC=<KERNEL SRC> pytest

Fair point.

> I don't feel like it is slowing anybody down. But since vmtest looks promising,
> maybe I can prepare a reverse logic. What I mean is I will prepare script which
> helps prepare the environment, but the default will be to use "locally installed
> stuff" when people just run "make -C tools/testing/selftests".

Let's keep it as is. 10sec for automated run is fine.