linux-kernel - Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <bf42489f-4a86-4717-b367-d8be877b3036@sirena.org.uk>
Date: Wed, 23 Oct 2024 22:24:29 +0100
From: Mark Brown <broonie@...nel.org>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: paulmck@...nel.org, Christoph Hellwig <hch@...radead.org>,
	Sasha Levin <sashal@...nel.org>, Kees Cook <kees@...nel.org>,
	ksummit@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: linus-next: improving functional testing for to-be-merged pull
 requests

On Wed, Oct 23, 2024 at 11:06:59AM -0700, Linus Torvalds wrote:

> And yes, I know some people do functional testing on linux-next
> already. The message at the maintainer summit was a bit mixed with
> some people saying linux-next tends to work even for that, others
> saying it's often too broken to be useful.

It very much depends on what you're trying to get out of the testing -
-next does work well most of the time, but it will absolutely just blow
up catastrophically on you from time time to time so you have to be
prepared to cope with loosing some or all of your coverage sometimes.
Usually anything major gets fixed fairly promptly, but sometimes you'll
be missing coverage for extended periods especially if it's something
like a more niche platform that's been broken or there's some problem
getting people to actually apply the fixes.  Submaintainer trees that
people don't want to add to -next can be an issue too.

You're also going to run into issues that are nothing to do with
whatever you're actually working on yourself and need to consider what
you're covering based on your tolerance for dealing with that.  The rate
of change can also be an issue if the tests you're intersted in are
expensive.  OTOH if you're doing things that are likely to be affected
by changes in a broad set of trees (eg, maintaining some embedded
platform where you care about all the various subsystems breaking
platform specific drivers) it can be a lot easier to cover -next rather
than all the individual subsystems.

Download attachment "signature.asc" of type "application/pgp-signature" (489 bytes)