linux-kernel - Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZyAUO0b3z_f_kVnj@sashalap>
Date: Mon, 28 Oct 2024 18:46:19 -0400
From: Sasha Levin <sashal@...nel.org>
To: Christoph Hellwig <hch@...radead.org>
Cc: Kees Cook <kees@...nel.org>, torvalds@...ux-foundation.org,
	ksummit@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: linus-next: improving functional testing for to-be-merged pull
 requests

On Mon, Oct 21, 2024 at 11:48:34PM -0700, Christoph Hellwig wrote:
>On Mon, Oct 21, 2024 at 09:54:53PM -0700, Kees Cook wrote:
>> For example, for a given PR, the bot can report:
>>
>> - Were the patches CCed to a mailing list?
>> - A histogram of how long the patches were in next (to show bake times)
>> - Are any patches associated with test failures? (0day and many other
>> CIs are already running tests against -next; parse those reports)
>>
>> We could have a real pre-submit checker! :)
>
>That would be very useful.  Items 1 and 2 should be trivial, 3 would
>require a bit of work but would still be very useful.

If you've been following so far, there is a bot that is capable of doing
most of the above
(https://git.kernel.org/pub/scm/linux/kernel/git/sashal/next-analysis.git/).

Here's a histogram that describes v6.12-rc4..v6.12-rc5 as far as how
long commits spent in -next:

Days in linux-next:
----------------------------------------
  0 | +++++++++++++++++++++++++++++++++++++++++++++++++ (89)
<1 | +++++++++++ (21)
  1 | +++++++++++ (21)
  2 | +++++++++++++++++++++++++ (45)
  3 | ++++++++++++++ (25)
  4 | +++++ (10)
  5 |
  6 | + (2)
  7 |
  8 | + (3)
  9 | ++ (4)
10 |
11 | +++ (6)
12 |
13 |
14+| ++++++++ (15)

This is where I think the value of linus-next comes during the -rc
cycles: the (89 + 21) commits that haven't gone through the -next
workflow before being pulled. I'm not looking to delay the process and
add latency, I'm looking to plug a hole where code would flow directly
to Linus's tree bypassing -next.

With linus-next, we can at least squeeze in build tests as well as some
rudimentary testing if we get a few hours before Linus pulls (and we
usually do).

-- 
Thanks,
Sasha