linux-kernel - Re: [PATCH] tags: much faster, parallel "make tags"

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:	Wed, 15 Apr 2015 15:41:28 +0200
From:	Michal Marek <mmarek@...e.cz>
To:	Alexey Dobriyan <adobriyan@...il.com>
CC:	akpm@...ux-foundation.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] tags: much faster, parallel "make tags"

On 2015-04-15 15:24, Michal Marek wrote:
> On 2015-04-14 19:20, Alexey Dobriyan wrote:
>> ctags is single-threaded program. Split list of files to be tagged into
>> equal parts, 1 part for each CPU and then merge the results.
>>
>> Speedup on one 2-way box I have is ~143 s => ~99 s (-31%).
>> On another 4-way box: ~120 s => ~65 s (-46%!).
>>
>> Resulting "tags" files aren't byte-for-byte identical because ctags
>> program numbers anon struct and enum declarations with "__anonNNN"
>> symbols. If those lines are removed, "tags" file becomes byte-for-byte
>> identical with those generated with current code.
>>
>> Signed-off-by: Alexey Dobriyan <adobriyan@...il.com>
>> ---
>>
>>  scripts/tags.sh |   34 ++++++++++++++++++++++++++++++++--
>>  1 file changed, 32 insertions(+), 2 deletions(-)
>>
>> --- a/scripts/tags.sh
>> +++ b/scripts/tags.sh
>> @@ -152,7 +152,24 @@ dogtags()
>>  
>>  exuberant()
>>  {
>> -	all_target_sources | xargs $1 -a                        \
>> +	NR_CPUS=1
>> +	if [ -e /proc/cpuinfo ]; then
>> +		NR_CPUS=$(grep -e '^processor	: ' /proc/cpuinfo | wc -l)
>> +	fi
> 
> I wonder if we should rather respect the -j option to make here. But
> then most people probably won't realize that make tags is parallel and
> will not use -j when generating tags. So let's leave it as is.

I meant, leave the concept as is, but fix the detection of the number of
cpus.


>> +	# write header
>> +	$1 -f tags /dev/null
>> +	# remove header
>> +	for i in .make-tags.*; do
>> +		sed -i -e '/^!/d' $i
>> +	done
>> +	sort .make-tags.* >>tags
> 
> The hardcoded "tags" filename will break 'make TAGS' when using
> exuberant ctags via an 'etags' symlink.

Additionally, the TAGS file must not be sorted.

Michal
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/