Date: Fri, 28 Jun 2024 11:22:49 +0200
From: Laura Nao <laura.nao@...labora.com>
To: kernelci@...ts.linux.dev
Cc: linux-kselftest@...r.kernel.org,
	linux-kernel@...r.kernel.org
Subject: Testing Quality Call notes - 2024-06-27

Hello,

KernelCI is hosting a bi-weekly call on Thursday to discuss improvements 
to existing upstream tests, the development of new tests to increase 
kernel testing coverage, and the enablement of these tests in KernelCI. 
In recent months, we at Collabora have focused on various kernel areas,
assessing the tests already available upstream and contributing patches 
to make them easily runnable in CIs.

Below is a list of the tests we've been working on and their latest 
status updates, as discussed in the last meeting held on 2024-06-27:

*USB/PCI devices kselftest*

- Upstream test to detect unprobed devices on discoverable buses:
  https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=dacf1d7a78bf8a131346c47bfba7fe1f3ff44beb 
- Kernel patches to allow running the test on more platforms on KernelCI 
  were merged:
  https://lore.kernel.org/all/20240613-kselftest-discoverable-probe-mt8195-kci-v1-0-7b396a9b032d@collabora.com
- Waiting for KernelCI PRs to be merged:
  https://github.com/kernelci/kernelci-core/pull/2577 and https://github.com/kernelci/kernelci-pipeline/pull/642

*Error log test*

- Proposing new kselftest to report device log errors: 
  https://lore.kernel.org/all/20240423-dev-err-log-selftest-v1-0-690c1741d68b@collabora.com/
- Currently fixing test failures in KernelCI

*Suspend/resume in cpufreq kselftest*

- Enabling suspend/resume test within the cpufreq kselftest in KernelCI
- Parameter support for running subtests in a kselftest was merged: 
  https://github.com/Linaro/test-definitions/pull/511
- Added rtcwake support in the test to enable automated resume, currently 
  testing/debugging solution

*Boot time test*

- Investigating possibility of adding new test upstream to measure the 
  kernel boot time and detect regressions
- Currently looking into boot tracing with ftrace events and kprobes 
  (see: https://www.kernel.org/doc/html/latest/trace/boottime-trace.html)
- Idea for potential kselftest: insert explicit tracepoints in strategic 
  places, let the user configure which times to measure. The test could 
  provide a bootconfig file and a fragment to enable the required configs.
  This could be an alternative to using external tools (e.g. grabserial 
  w/ early serial port init).
- Need a list of functions to track in order to measure key metrics 
  (e.g. device tree overhead, probe overhead, module load overhead)
- Identify key drivers that need to be loaded early, for potentially 
  supporting a two-phase boot: (1) time-critical, and (2) rest of the 
  system

*Other interesting updates*

- Flaky serial on sc7180 was recently fixed: 
  https://github.com/kernelci/kernelci-project/issues/380 and https://lore.kernel.org/all/20240610222515.3023730-1-dianders@chromium.org/#t

*Strategy for test enablement in KernelCI*

- Guidance on test quality: KernelCI should set the standard for test 
  quality, providing guidance on which tests to enable from various 
  projects (e.g., kselftest, LTP). By doing so, KernelCI can serve as a 
  model for other CI systems.
- Develop mechanisms to automatically detect which tests should run on a
  specific platform
- Embed metadata in the test themselves to facilitate the test selection
  process
- Leverage device tree info to determine the appropriate tests for each 
  platform

Please reply to this thread if you'd like to join the call or discuss 
any of the topics further. We look forward to collaborating with the 
community to improve upstream tests and expand coverage to more areas 
of interest within the kernel.

Best regards,

Laura Nao

Powered by blists - more mailing lists