linux-kernel - Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <xhjzj6kfgg2dxq6swurwaeyzqtd2sl4dat5pzg6jolirw5og6z@bmwdcuwsf2bv>
Date: Mon, 21 Oct 2024 14:36:39 -0400
From: "Liam R. Howlett" <Liam.Howlett@...cle.com>
To: Sasha Levin <sashal@...nel.org>
Cc: torvalds@...ux-foundation.org, ksummit@...ts.linux.dev,
        linux-kernel@...r.kernel.org
Subject: Re: linus-next: improving functional testing for to-be-merged pull
 requests

* Sasha Levin <sashal@...nel.org> [241021 12:32]:
> Hi folks,
> 
> The linux-next tree we all know and love is widely used by the kernel
> community for integration work. It offers several advantages:
> 
> 	1. Early detection of conflicts between matinainer trees
> 
> 	2. Catching most new build errors/warnings

Would it be difficult to catch branches that change things outside their
scope without correct SOB/RB/Acks?  Asking for a friend...

> 
> However, it faces significant testing challenges:
> 
> 	1. Contains a mix of "ready-to-go" code and experimental additions
> 
> 	2. A single "bad" piece of code can affect testing of everything else
> 
> 	3. Low barrier of entry, encouraging inclusion over exclusion
> 
> 	4. While linux-next offers early conflict resolution and
> 	identifies build issues, it is very difficult to actually test
> 	due to the abundance of runtime issues it tends to have
> 
> These factors combine to make linux-next a valuable tool for integration
> but problematic for comprehensive testing.
> 
> During the Maintainer's Summit, Linus Torvalds expressed concerns about
> the quality of testing that code receives before he pulls it. The
> subsequent discussion side-tracked to the testability of linux-next, but
> we didn't directly address Linus's original concern about pre-pull
> testing quality.
> 
> In an attempt to address the concerns, we're trying out a new "linus-next"

The names are really close, could it be something that's more than one
character different?

linus-pulled, linux-pending, linux-pr or something?  They are also the
same length, which adds to the parsing challenge on both typing and
reading.  I'm thinking I'll get emails about the wrong branch or send
them with the wrong branch specified - especially pre-coffee.

> tree is being created and maintained with the following characteristics:
> 
> 	1. Composed of pull requests sent directly to Linus
> 
> 	2. Contains branches destined for imminent inclusion by Linus
> 
> 	3. Higher code quality expectation (these are pull requests that
> 	maintainers expect Linus to pull)
> 
> 	4. Continuous tree (not daily tags like in linux-next),
> 	facilitating easier bisection
> 
> The linus-next tree aims to provide a more stable and testable
> integration point compared to linux-next, addressing the runtime issues
> that make testing linux-next challenging and focusing on code that's
> about to be pulled by Linus.

What about the people who send them late?  I know people get told not to
do that, but some people still do.  Those late pull requests would
potentially invalidate a lot of the testing in this scenario.

For example, if there was a late mm change, then anything using mm could
be affected.  That's a large subset.

Is there any cut-off time for testing?

Thanks,
Liam