linux-kernel - Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <d75c9c2f-353f-464c-89d3-8c18dbfb4770@leemhuis.info>
Date: Tue, 29 Oct 2024 09:10:25 +0100
From: Thorsten Leemhuis <linux@...mhuis.info>
To: Sasha Levin <sashal@...nel.org>, Christoph Hellwig <hch@...radead.org>
Cc: Kees Cook <kees@...nel.org>, torvalds@...ux-foundation.org,
 ksummit@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: linus-next: improving functional testing for to-be-merged pull
 requests

On 28.10.24 23:46, Sasha Levin wrote:
> On Mon, Oct 21, 2024 at 11:48:34PM -0700, Christoph Hellwig wrote:
>> On Mon, Oct 21, 2024 at 09:54:53PM -0700, Kees Cook wrote:
>>> For example, for a given PR, the bot can report:
>>>
>>> - Were the patches CCed to a mailing list?
>>> - A histogram of how long the patches were in next (to show bake times)
>>> - Are any patches associated with test failures? (0day and many other
>>> CIs are already running tests against -next; parse those reports)
>>>
>>> We could have a real pre-submit checker! :)
>>
>> That would be very useful.  Items 1 and 2 should be trivial, 3 would
>> require a bit of work but would still be very useful.
> 
> If you've been following so far, there is a bot that is capable of doing
> most of the above
> (https://git.kernel.org/pub/scm/linux/kernel/git/sashal/next-
> analysis.git/).
> 
> Here's a histogram that describes v6.12-rc4..v6.12-rc5 as far as how
> long commits spent in -next:

I took a quick look at that tree and histo.sh that lead to a few
questions here the code had no obvious answers to (or I missed them due
to the "quick" aspect):

* How does histo.sh handle changes where the commit-id changed between
the first time in -next and their merge into Linus' tree (while the
patch itself did not change)? For example due to a rebase or workflows
where the commit-id changes regularly, such as those used by the
bluetooth tree (for -fixes, as it queues them in their -next branch
first) or the -mm tree (for most of it iirc -- this made things hard in
a script of mine that looks up the arrival in -next)?

* Do those lore scripts detect if a committer adjusted the subject of a
patch that has been on lore?

* How do the scripts handle patches that changed a lot while they were
in -next? I know of one subsystem that regularly drops whole patch-sets
from their trees included in -next to replace them with newer versions
of said patch-sets -- and then the timer maybe should restart.

> This is where I think the value of linus-next comes during the -rc
> cycles: the (89 + 21) commits that haven't gone through the -next
> workflow before being pulled. 
>
> I'm not looking to delay the process and
> add latency, I'm looking to plug a hole where code would flow directly
> to Linus's tree bypassing -next.

Overall after all the discussions in this thread I still fail to see why
we need a new tree for that. Why not make pending-fixes a bit more
prominent while motivating maintainers to have proper -fixes branches
included there?

Ciao, Thorsten