linux-kernel - Re: [PATCH] scripts: stable: add script to validate backports

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Sat, 27 Mar 2021 14:46:52 +0100
From:   Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:     Tom Saeger <tom.saeger@...cle.com>
Cc:     Nick Desaulniers <ndesaulniers@...gle.com>,
        Sasha Levin <sashal@...nel.org>,
        clang-built-linux <clang-built-linux@...glegroups.com>,
        LKML <linux-kernel@...r.kernel.org>,
        "# 3.4.x" <stable@...r.kernel.org>
Subject: Re: [PATCH] scripts: stable: add script to validate backports

On Fri, Mar 26, 2021 at 03:03:35PM -0600, Tom Saeger wrote:
> On Wed, Mar 24, 2021 at 10:55:27AM +0100, Greg Kroah-Hartman wrote:
> > On Tue, Mar 23, 2021 at 01:28:38PM -0700, Nick Desaulniers wrote:
> > > On Tue, Mar 23, 2021 at 12:05 PM Greg Kroah-Hartman
> > > <gregkh@...uxfoundation.org> wrote:
> > > >
> > > > The only time git gets involved is when we do a -rc release or when we
> > > > do a "real" release, and then we use 'git quiltimport' on the whole
> > > > stack.
> > > >
> > > > Here's a script that I use (much too slow, I know), for checking this
> > > > type of thing and I try to remember to run it before every cycle of -rc
> > > > releases:
> > > >         https://github.com/gregkh/commit_tree/blob/master/find_fixes_in_queue
> > > >
> > > > It's a hack, and picks up more things than is really needed, but I would
> > > > rather it error on that side than the other.
> > > 
> > > Yes, my script is similar.  Looks like yours also runs on a git tree.
> > > 
> > > I noticed that id_fixed_in runs `git grep -l --threads=3 <sha>` to
> > > find fixes; that's neat, I didn't know about `--threads=`.  I tried it
> > > with ae46578b963f manually:
> > > 
> > > $ git grep -l --threads=3 ae46578b963f
> > > $
> > > 
> > > Should it have found a7889c6320b9 and 773e0c402534?  Perhaps `git log
> > > --grep=<sha>` should be used instead?  I thought `git grep` only greps
> > > files in the archive, not commit history?
> > 
> > Yes, it does only grep the files in the archive.
> > 
> > But look closer at the archive that this script lives in :)
> > 
> > This archive is a "blown up" copy of the Linux kernel tree, with one
> > file per commit.  The name of the file is the commit id, and the content
> > of the file is the changelog of the commit itself.
> > 
> > So it's a hack that I use to be able to simply search the changelogs of
> > all commits to find out if they have a "Fixes:" tag with a specific
> > commit id in it.
> > 
> > So in your example above, in the repo I run it and get:
> > 
> > ~/linux/stable/commit_tree $ git grep -l --threads=3 ae46578b963f
> > changes/5.2/773e0c40253443e0ce5491cb0e414b62f7cc45ed
> > ids/5.2
> > 
> > Which shows me that in commit 773e0c402534 ("afs: Fix
> > afs_xattr_get_yfs() to not try freeing an error value") in the kernel
> > tree, it has a "Fixes:" tag that references "ae46578b963f".
> > 
> > It also shows me that commit ae46578b963f was contained in the 5.2
> > kernel release, as I use the "ids/" subdirectory here for other fast
> > lookups (it's a tiny bit faster than 'git describe --contains').
> > 
> > I don't know how your script is walking through all possible commits to
> > see if they are fixing a specific one, maybe I should look and see if
> > it's doing it better than my "git tree/directory as a database hack"
> > does :)
> 
> FWIW,
> 
> I had a need for something similar and found `git rev-list --grep` provided fastest
> results.  Does not provide for the "ids/" hack though...
> 
> ❯ N="ae46578b963f"; git rev-list --grep="${N}" "${N}..upstream/master" | while read -r hid ; do git log -n1 "${hid}" | grep -F "${N}" | sed "s#^#${hid} #"; done
> a7889c6320b9200e3fe415238f546db677310fa9     Fixes: ae46578b963f ("afs: Get YFS ACLs and information through xattrs")
> 773e0c40253443e0ce5491cb0e414b62f7cc45ed     Fixes: ae46578b963f ("afs: Get YFS ACLs and information through xattrs")
> 
> ❯ N="a7889c6320b9"; git rev-list --grep="${N}" "${N}..stable/linux-5.4.y" | while read -r hid ; do git log -n1 "${hid}" | grep -F "${N}" | sed "s#^#${hid} #"; done
> 6712b7fcef9d1092e99733645cf52cfb3d482555     commit a7889c6320b9200e3fe415238f546db677310fa9 upstream.
> 
> ❯ N="ae46578b963f"; git rev-list --grep="${N}" "${N}..stable/linux-5.4.y" | while read -r hid ; do git log -n1 "${hid}" | grep -F "${N}" | sed "s#^#${hid} #"; done
> 6712b7fcef9d1092e99733645cf52cfb3d482555     Fixes: ae46578b963f ("afs: Get YFS ACLs and information through xattrs")
> 773e0c40253443e0ce5491cb0e414b62f7cc45ed     Fixes: ae46578b963f ("afs: Get YFS ACLs and information through xattrs")
> 
> 

Ah, I did not know about 'git rev-list --grep' thanks!  I'll play around
with that to see if it actually is any faster than my implementation...

thanks,

greg k-h