linux-kernel - Re: [PATCH v3 0/7] get_abi.pl: Check for missing symbols at the ABI specs

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 21 Sep 2021 18:52:42 +0200
From:   Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:     Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
Cc:     Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        linux-kernel@...r.kernel.org, Jonathan Corbet <corbet@....net>,
        Anton Vorontsov <anton@...msg.org>,
        Colin Cross <ccross@...roid.com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...nel.org>,
        Kees Cook <keescook@...omium.org>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>,
        Tony Luck <tony.luck@...el.com>, Yonghong Song <yhs@...com>,
        bpf@...r.kernel.org, netdev@...r.kernel.org
Subject: Re: [PATCH v3 0/7] get_abi.pl: Check for missing symbols at the ABI
 specs

On Sat, Sep 18, 2021 at 11:52:10AM +0200, Mauro Carvalho Chehab wrote:
> Hi Greg,
> 
> Add a new feature at get_abi.pl to optionally check for existing symbols
> under /sys that won't match a "What:" inside Documentation/ABI.
> 
> Such feature is very useful to detect missing documentation for ABI.
> 
> This series brings a major speedup, plus it fixes a few border cases when
> matching regexes that end with a ".*" or \d+.
> 
> patch 1 changes get_abi.pl logic to handle multiple What: lines, in
> order to make the script more robust;
> 
> patch 2 adds the basic logic. It runs really quicky (up to 2
> seconds), but it doesn't use sysfs softlinks.
> 
> Patch 3 adds support for parsing softlinks. It makes the script a
> lot slower, making it take a couple of minutes to process the entire
> sysfs files. It could be optimized in the future by using a graph,
> but, for now, let's keep it simple.
> 
> Patch 4 adds an optional parameter to allow filtering the results
> using a regex given by the user. When this parameter is used
> (which should be the normal usecase), it will only try to find softlinks
> if the sysfs node matches a regex.
> 
> Patch 5 improves the report by avoiding it to ignore What: that
> ends with a wildcard.
> 
> Patch 6 is a minor speedup.  On a Dell Precision 5820, after patch 6, 
> results are:
> 
> 	$ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols
> 
> 	real	2m35.563s
> 	user	2m34.346s
> 	sys	0m1.220s
> 	7595 undefined
> 	896 undefined_symbols
> 
> Patch 7 makes a *huge* speedup: it basically switches a linear O(n^3)
> search for links by a logic which handle symlinks using BFS. It
> also addresses a border case that was making 'msi-irqs/\d+' regex to
> be misparsed. 
> 
> After patch 7, it is 11 times faster:
> 
> 	$ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols
> 
> 	real	0m14.137s
> 	user	0m12.795s
> 	sys	0m1.348s
> 	7030 undefined
> 	794 undefined_symbols
> 
> (the difference on the number of undefined symbols are due to the fix for
> it to properly handle 'msi-irqs/\d+' regex)
> 
> -
> 
> While this series is independent from Documentation/ABI changes, it
> works best when applied from this tree, which also contain ABI fixes
> and a couple of additions of frequent missed symbols on my machine:
> 
>     https://git.kernel.org/pub/scm/linux/kernel/git/mchehab/devel.git/log/?h=get_undefined_abi_v3

I've taken all of these, but get_abi.pl seems to be stuck in an endless
loop or something.  I gave up and stopped it after 14 minutes.  It had
stopped printing out anything after finding all of the pci attributes
that are not documented :)

Anything I can do to help debug this?

thanks,

greg k-h