lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
Open Source and information security mailing list archives
| ||
|
Message-ID: <CH3PR11MB84141E0EDA588B84F7E10F71E31EA@CH3PR11MB8414.namprd11.prod.outlook.com> Date: Mon, 21 Aug 2023 09:32:56 +0000 From: "Michalik, Michal" <michal.michalik@...el.com> To: Jakub Kicinski <kuba@...nel.org> CC: "netdev@...r.kernel.org" <netdev@...r.kernel.org>, "vadim.fedorenko@...ux.dev" <vadim.fedorenko@...ux.dev>, "jiri@...nulli.us" <jiri@...nulli.us>, "Kubalewski, Arkadiusz" <arkadiusz.kubalewski@...el.com>, "jonathan.lemon@...il.com" <jonathan.lemon@...il.com>, "pabeni@...hat.com" <pabeni@...hat.com>, poros <poros@...hat.com>, "Olech, Milena" <milena.olech@...el.com>, mschmidt <mschmidt@...hat.com>, "linux-clk@...r.kernel.org" <linux-clk@...r.kernel.org>, "bvanassche@....org" <bvanassche@....org> Subject: RE: [PATCH RFC net-next v1 2/2] selftests/dpll: add DPLL system integration selftests On 18 August 2023 11:08 PM CEST, Jakub Kicinski wrote: > > On Thu, 17 Aug 2023 17:22:09 +0200 Michal Michalik wrote: >> High level flow of DPLL subsystem integration selftests: >> (after running run_dpll_tests.sh or 'make -C tools/testing/selftests') >> 1) check if Python in correct version is installed, >> 2) create temporary Python virtual environment, >> 3) install all the required libraries, >> 4) run the tests, >> 5) do cleanup. > > How fragile do you reckon this setup will be? > I mean will it work reliably across distros and various VM setups? > I have tried writing tests based on ynl.py and the C codegen, and > I can't decide whether the python stuff is easy enough to deploy. > Much easier to scp over to the test host a binary based on the > C code. But typing tests in python is generally quicker... > What are your thoughts? > > Thanks for posting the tests! Hi Jakub, First of all - everything I'll write is just my opinion. While having quite a bit Python experience, I can't speak confidently about the target systems and hardware (architectures) it would be ran against and what might be possible problems around that. I need your help here. >From my point of view, all we need is a Python 3.7 and access to PyPI repositories. This version of Python is 5 years old, but if we need support of even older versions I can rewrite the code not to use dataclasses[1] so we could go with even older version. Do you think it is required to support platforms with no Python at all? Another requirement is the toolchain for building the module, but I assume if Python is not there in some embedded system - the build tools are also not there... I've seen other Python pytest code in the kernel repo - that is why I thought it might be a good idea to propose something like that. Your idea is also cool - binary is always superior. I am a strong advocate of taking into consideration not only deployment ease, but also maintenance and ease of read which might encourage community to help. I also see a benefit of showing the sample implementation (my tested "dummy module"). My deployment is automatic and does not leave any garbage in the system after tests (packages are installed into temporary virtual environment). In case any of the requirements are not met - tests are skipped. I've tested it both on real HW with some E810T cards installed and on fresh VM - seems to work fine. But that of course require further verification. Till now only Arek Kubalewski sucessfully gave those tests a shot. The biggest concern for me is the requirement of selftests[2]: "Don't take too long;" This approach is reloading the modules few times to check few scenarios. Also, the DPLL subsystem is being tested against multiple requests - so it takes some time to finish (not too long but is definitely not instant). If you asked me, I would consider those tests to be part of the kernel. I am not sure if they should be a part of selftests, though. Maybe a reasonable approach would be to have have my tests being a "thorough integration tests" and at the same time some limited tests shipped as a selftests binary? I can also parametrically limit the scope of my tests to run faster in selftests (e.g. only one scenario) and having possibility to run extensive tests on demand? Thanks, M^2 [1] https://docs.python.org/3/library/dataclasses.html [2] https://www.kernel.org/doc/html/v5.0/dev-tools/kselftest.html
Powered by blists - more mailing lists