Date:   Sat, 18 Sep 2021 11:52:10 +0200
From:   Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To:     Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc:     Mauro Carvalho Chehab <mchehab+huawei@...nel.org>,
        linux-kernel@...r.kernel.org, Jonathan Corbet <corbet@....net>,
        Anton Vorontsov <anton@...msg.org>,
        Colin Cross <ccross@...roid.com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...nel.org>,
        Kees Cook <keescook@...omium.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>,
        Tony Luck <tony.luck@...el.com>, Yonghong Song <yhs@...com>,
        bpf@...r.kernel.org, netdev@...r.kernel.org
Subject: [PATCH v3 0/7] get_abi.pl: Check for missing symbols at the ABI  specs

Hi Greg,

Add a new feature at get_abi.pl to optionally check for existing symbols
under /sys that won't match a "What:" inside Documentation/ABI.

Such feature is very useful to detect missing documentation for ABI.

This series brings a major speedup, plus it fixes a few border cases when
matching regexes that end with a ".*" or \d+.

patch 1 changes get_abi.pl logic to handle multiple What: lines, in
order to make the script more robust;

patch 2 adds the basic logic. It runs really quicky (up to 2
seconds), but it doesn't use sysfs softlinks.

Patch 3 adds support for parsing softlinks. It makes the script a
lot slower, making it take a couple of minutes to process the entire
sysfs files. It could be optimized in the future by using a graph,
but, for now, let's keep it simple.

Patch 4 adds an optional parameter to allow filtering the results
using a regex given by the user. When this parameter is used
(which should be the normal usecase), it will only try to find softlinks
if the sysfs node matches a regex.

Patch 5 improves the report by avoiding it to ignore What: that
ends with a wildcard.

Patch 6 is a minor speedup.  On a Dell Precision 5820, after patch 6, 
results are:

	$ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols

	real	2m35.563s
	user	2m34.346s
	sys	0m1.220s
	7595 undefined
	896 undefined_symbols

Patch 7 makes a *huge* speedup: it basically switches a linear O(n^3)
search for links by a logic which handle symlinks using BFS. It
also addresses a border case that was making 'msi-irqs/\d+' regex to
be misparsed. 

After patch 7, it is 11 times faster:

	$ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols

	real	0m14.137s
	user	0m12.795s
	sys	0m1.348s
	7030 undefined
	794 undefined_symbols

(the difference on the number of undefined symbols are due to the fix for
it to properly handle 'msi-irqs/\d+' regex)

-

While this series is independent from Documentation/ABI changes, it
works best when applied from this tree, which also contain ABI fixes
and a couple of additions of frequent missed symbols on my machine:

    https://git.kernel.org/pub/scm/linux/kernel/git/mchehab/devel.git/log/?h=get_undefined_abi_v3

-

v3:
  - Fixed parse issues with 'msi-irqs/\d+' regex;
  - Added a BFS graph logic to solve symlinks at sysfs;

v2:
  - multiple What: for the same description are now properly handled;
  - some special cases are now better handled;
  - some bugs got fixed.

The full series, with the ABI changes and some ABI improvements can be found
at:
	https://git.kernel.org/pub/scm/linux/kernel/git/mchehab/devel.git/commit/?h=get_undefined&id=1838d8fb149170f6c19feda0645d6c3157f46f4f



Mauro Carvalho Chehab (7):
  scripts: get_abi.pl: Better handle multiple What parameters
  scripts: get_abi.pl: Check for missing symbols at the ABI specs
  scripts: get_abi.pl: detect softlinks
  scripts: get_abi.pl: add an option to filter undefined results
  scripts: get_abi.pl: don't skip what that ends with wildcards
  scripts: get_abi.pl: Ignore fs/cgroup sysfs nodes earlier
  scripts: get_abi.pl: add a graph to speedup the undefined algorithm

 scripts/get_abi.pl | 327 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 320 insertions(+), 7 deletions(-)

-- 
2.31.1

Powered by blists - more mailing lists