netdev - RE: [PATCH RFC net-next v4 2/2] selftests/dpll: add DPLL system integration selftests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date: Fri, 1 Dec 2023 18:33:11 +0000
From: "Michalik, Michal" <michal.michalik@...el.com>
To: Jakub Kicinski <kuba@...nel.org>
CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>,
	"vadim.fedorenko@...ux.dev" <vadim.fedorenko@...ux.dev>, "Kubalewski,
 Arkadiusz" <arkadiusz.kubalewski@...el.com>, "jonathan.lemon@...il.com"
	<jonathan.lemon@...il.com>, "pabeni@...hat.com" <pabeni@...hat.com>, poros
	<poros@...hat.com>, "Olech, Milena" <milena.olech@...el.com>, mschmidt
	<mschmidt@...hat.com>, "linux-clk@...r.kernel.org"
	<linux-clk@...r.kernel.org>, "bvanassche@....org" <bvanassche@....org>,
	"davem@...emloft.net" <davem@...emloft.net>, "edumazet@...gle.com"
	<edumazet@...gle.com>
Subject: RE: [PATCH RFC net-next v4 2/2] selftests/dpll: add DPLL system
 integration selftests

On 1 December 2023 7:51 AM CET, Jakub Kicinski wrote:
> 
> On Thu, 30 Nov 2023 17:46:37 +0000 Michalik, Michal wrote:
>> > Did you try to run it in vmtest or virtme-ng?
>> > https://www.youtube.com/watch?v=NT-325hgXjY
>> > https://lpc.events/event/17/contributions/1506/attachments/1143/2441/virtme-ng.pdf
>> > 
>> > I'm thinking of using those for continuous testing, curious all 
>> > the Python setup works okay with them.  
>> 
>> Very interesting idea, I didn't try to use those - will get familiar with that and
>> see if I can make any improvements to go with vmtest/virtme-ng before I will send
>> out the RFC v5.
> 
> LMK how it goes. I tried using both today and they work fine if I let
> them build the kernel, but if I tried to use my own kernel build they
> just hang :(
> 

That looks really promising - great idea. I tried only vmtest today, and my tests
work kind of flawless with my own built kernel (nested VMs):
  $ vmtest -k /home/net-next/vmlinux "modprobe netdevsim && KSRC=/home/net-next/ pytest"
  => vmlinux
  ===> Booting
  ===> Setting up VM
  ===> Running command
  ============================= test session starts ==============================
  platform linux -- Python 3.9.16, pytest-7.4.3, pluggy-1.3.0
  rootdir: /home/net-next/tools/testing/selftests/drivers/net/netdevsim/dpll
  collected 91 items  

  test_dpll.py ........................................................... [ 64%]
  ................................                                         [100%]

  ============================= 91 passed in 10.54s ==============================

I will try to take a look at virtme-ng next week, but to be frank I already like
the vmtest.

>> > Did you see what the sdsi test does? It seems to assume everything 
>> > is installed locally, without the venv. I wonder if that may be simpler
>> > to get going with vmtest?  
>> 
>> To be honest I did not see that. I agree that this is a simpler solution, but I am
>> not sure if that is not "too simple". What I mean, I'm not sure who wrote the sdsi
>> tests, but maybe they were not aware about the Python best practices? Python used
>> to be my first language, and I would vote for using the venvs if you asked me.
>> I understand that it haven't been done before, but we are here to try to improve
>> the things, yes? 
> 
> I think I already asked how long the setup takes but my only concern 
> is that the setup will be slower, and less useful during development.
>

I wanted for "run_dpll_test.sh" to be userfriendly even for people who does not
have a clue how python/pytest works. If somebody is developing tests, I assume
he/she knows what she is doing and is using own environment either way, like
venvs with additional Python debug tools and direct pytest in tests directory:
  KSRC=<KERNEL SRC> pytest

I don't feel like it is slowing anybody down. But since vmtest looks promising,
maybe I can prepare a reverse logic. What I mean is I will prepare script which
helps prepare the environment, but the default will be to use "locally installed
stuff" when people just run "make -C tools/testing/selftests".

>> Of course if you outvote me, I won't act as Tadeusz Rejtan in
>> Matejko's painting "The Fall of Poland" and just remove the virtual environments. :)
> 
> :D
> The infallible strategy of showing a nipple.
> https://www.youtube.com/watch?v=lY0V65YWEIA&t=50s
>

Good one! :D