linux-kernel - Re: Re: linus-next: improving functional testing for to-be-merged pull requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <gwcovjajuiiyw4v57jafnfjquoxtnlw23kwyzn2nla42zia6pp@ic3eoqom2wwo>
Date: Mon, 25 Nov 2024 21:05:05 +0100
From: Joel Granados <joel.granados@...nel.org>
To: Sasha Levin <sashal@...nel.org>
Cc: Christoph Hellwig <hch@...radead.org>, Kees Cook <kees@...nel.org>, 
	torvalds@...ux-foundation.org, ksummit@...ts.linux.dev, linux-kernel@...r.kernel.org
Subject: Re: Re: linus-next: improving functional testing for to-be-merged
 pull requests

On Mon, Oct 28, 2024 at 06:46:19PM -0400, Sasha Levin wrote:
> On Mon, Oct 21, 2024 at 11:48:34PM -0700, Christoph Hellwig wrote:
> >On Mon, Oct 21, 2024 at 09:54:53PM -0700, Kees Cook wrote:
> >> For example, for a given PR, the bot can report:
> >>
> >> - Were the patches CCed to a mailing list?
> >> - A histogram of how long the patches were in next (to show bake times)
> >> - Are any patches associated with test failures? (0day and many other
> >> CIs are already running tests against -next; parse those reports)
> >>
> >> We could have a real pre-submit checker! :)
> >
> >That would be very useful.  Items 1 and 2 should be trivial, 3 would
> >require a bit of work but would still be very useful.
> 
> If you've been following so far, there is a bot that is capable of doing
> most of the above
> (https://git.kernel.org/pub/scm/linux/kernel/git/sashal/next-analysis.git/).
> 
> Here's a histogram that describes v6.12-rc4..v6.12-rc5 as far as how
> long commits spent in -next:
> 
> Days in linux-next:
> ----------------------------------------
>   0 | +++++++++++++++++++++++++++++++++++++++++++++++++ (89)
> <1 | +++++++++++ (21)
>   1 | +++++++++++ (21)
>   2 | +++++++++++++++++++++++++ (45)
>   3 | ++++++++++++++ (25)
>   4 | +++++ (10)
>   5 |
>   6 | + (2)
>   7 |
>   8 | + (3)
>   9 | ++ (4)
> 10 |
> 11 | +++ (6)
> 12 |
> 13 |
> 14+| ++++++++ (15)

I kept working on this for this release and ended up with something that answers
the question how much time has my PR been in linux-next. Not sure if it is
useful to anyone, but decided to share it just in case.

# The command
``` bash
./pr-hist.sh -d ./db --linux-next ~/src/linux-next --pr /tmp/pull_request.txt
```
* here [1] are the script sources
* pull_request.txt is a file that contains a PR created with the `git
  request-pull` command.

# The output

``` bash
                         Histogram of Counts by Date
120 +-------------------------------------------------------------------+
    |                                                     * *           |
100 |                                                     * *           |
    |                                                  ** * *           |
 80 |                                                * ** * *           |
 60 |                                        * *     * ** * *           |
    |                                    * * * *     * ** * *           |
 40 |                        * * *     * * * * *     * ** * *           |
    |           * * *    * * * * *     * * * * *     * ** * *           |
 20 |           * * *    * * * * *     * * * * *     * ** * *           |
    |           * * *    * * * * *     * * * * *     * ** * *           |
  0 +-------------------------------------------------------------------+
  1017          1024         1031          1107         1114          1121
                                    Date
```

# Command help
``` bash
  Usage: pr-hist.sh ARGS [OPTIONS]

  Lists the databases where the <commit_ids> are found

  Args: [-c <list> | -p <pr>] -d <db_dir>
    -c, --commit-ids <list>       Comma separated list of commits to query
    -p, --pr <pr>                 Pull request file (generated by request-pull)
    -d, --db <db_dir>             Database directory

  Options:
    -n, --linux-next <next_dir>   The linux-next directory
    -h, --help                    Display this help
```

Best

[1] https://git.kernel.org/pub/scm/linux/kernel/git/joel.granados/next-analysis.git/tree/pr-hist.sh?h=jag/master

-- 

Joel Granados