linux-kernel - Re: [PATCH 0/8] (REBASED) get_abi.pl undefined: improve precision and performance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 27 Sep 2021 10:55:53 +0200
From:   Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>
Cc:     Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        linux-kernel@...r.kernel.org, Jonathan Corbet <corbet@....net>
Subject: Re: [PATCH 0/8] (REBASED) get_abi.pl undefined: improve precision
 and performance

Em Thu, 23 Sep 2021 19:13:04 +0200
Greg Kroah-Hartman <gregkh@...uxfoundation.org> escreveu:

> On Thu, Sep 23, 2021 at 05:41:11PM +0200, Mauro Carvalho Chehab wrote:
> > Hi Greg,
> > 
> > As requested, this is exactly the same changes, rebased on the top of
> > driver-core/driver-core-next.
> > 
> > -
> > 
> > It follows a series of improvements for get_abi.pl. it is on the top of driver-core/driver-core-next.
> > 
> > With such changes, on my development tree, the script is taking 6 seconds to run 
> > on my desktop:
> > 
> > 	$ !1076
> > 	$ time ./scripts/get_abi.pl undefined |sort >undefined_after && cat undefined_after| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined_after undefined_symbols
> > 
> > 	real	0m6,292s
> > 	user	0m5,640s
> > 	sys	0m0,634s
> > 	  6838 undefined_after
> > 	   808 undefined_symbols
> > 	  7646 total
> > 
> > And 7 seconds on a Dell Precision 5820:
> > 
> > 	$ time ./scripts/get_abi.pl undefined |sort >undefined && cat undefined| perl -ne 'print "$1\n" if (m#.*/(\S+) not found#)'|sort|uniq -c|sort -nr >undefined_symbols; wc -l undefined; wc -l undefined_symbols
> > 
> > 	real	0m7.162s
> > 	user	0m5.836s
> > 	sys	0m1.329s
> > 	6548 undefined
> > 	772 undefined_symbols
> > 
> > Both tests were done against this tree (based on today's linux-next):
> > 
> > 	$ https://git.kernel.org/pub/scm/linux/kernel/git/mchehab/devel.git/log/?h=get_abi_undefined-latest
> > 
> > It should be noticed that, as my tree has several ABI fixes,  the time to run the
> > script is likely less than if you run on your tree, as there will be less symbols to
> > be reported, and the algorithm is optimized to reduce the number of regexes
> > when a symbol is found.
> > 
> > Besides optimizing and improving the seek logic, this series also change the
> > debug logic. It how receives a bitmap, where "8" means to print the regexes
> > that will be used by "undefined" command:
> > 
> > 	$ time ./scripts/get_abi.pl undefined --debug 8 >foo
> > 	real	0m17,189s
> > 	user	0m13,940s
> > 	sys	0m2,404s
> > 
> > 	$wc -l foo
> > 	18421939 foo
> > 
> > 	$ cat foo
> > 	...
> > 	/sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/in_voltage.*_scale_available$)$/
> > 	/sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/out_voltage.*_scale_available$)$/
> > 	/sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/out_altvoltage.*_scale_available$)$/
> > 	/sys/kernel/kexec_crash_loaded =~ /^(?^:^/sys/.*/iio\:device.*/in_pressure.*_scale_available$)$/
> > 	...
> > 
> > On other words, on my desktop, the /sys match is performing >18M regular 
> > expression searches, which takes 6,2 seconds (or 17,2 seconds, if debug is 
> > enabled and sent to an area on my nvme storage).  
> 
> Better, it's down to 10 minutes on my machine now:
> 
> 	real	10m39.218s
> 	user	10m37.742s
> 	sys	0m0.775s

A lot better, but not clear why it is still taking ~40x more than here...
It could well be due to the other ABI changes yet to be applied
(I'll submit it probably later today), but it could also be related to
something else. Could this be due to disk writes?

Thanks,
Mauro