linux-kernel - Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZxeEA6i_xfBMxJm4@sashalap>
Date: Tue, 22 Oct 2024 06:52:51 -0400
From: Sasha Levin <sashal@...nel.org>
To: Christoph Hellwig <hch@...radead.org>
Cc: Kees Cook <kees@...nel.org>, torvalds@...ux-foundation.org,
	ksummit@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: linus-next: improving functional testing for to-be-merged pull
 requests

On Mon, Oct 21, 2024 at 11:48:34PM -0700, Christoph Hellwig wrote:
>On Mon, Oct 21, 2024 at 09:54:53PM -0700, Kees Cook wrote:
>> For example, for a given PR, the bot can report:
>>
>> - Were the patches CCed to a mailing list?
>> - A histogram of how long the patches were in next (to show bake times)
>> - Are any patches associated with test failures? (0day and many other
>> CIs are already running tests against -next; parse those reports)
>>
>> We could have a real pre-submit checker! :)
>
>That would be very useful.  Items 1 and 2 should be trivial, 3 would
>require a bit of work but would still be very useful.

So I've ended up writing something hacky that does (1) and (2) above, and
ran it on the existing (already merged) PRs on the mailing list to give
it a spin.

The script tripped on the very first PR it looked at:
https://lore.kernel.org/all/20241021171728.274997-1-pbonzini@redhat.com/

And in particular, this commit: afa9b48f327c ("KVM: arm64: Shave a few
bytes from the EL2 idmap code")

(sorry, not trying to pick on anyone/anything, just an example...)

The commit can't be found on lore.kernel.org, it was never in -next, and
yet Linus pulled it promptly without questioning anything.

So yes, we could argue that a bot sending out this information would
have informed Linus of this issue, but somehow I suspect that Linus is
not an idiot and he has enough scripts in place already to alert him of
these issues, which he chooses to ignore in some cases.

-- 
Thanks,
Sasha