linux-kernel - Re: [PATCH 12/12] docs: kdoc: Improve the output text accumulation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20250710125914.41fe8cf6@foz.lan>
Date: Thu, 10 Jul 2025 12:59:14 +0200
From: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To: Jonathan Corbet <corbet@....net>
Cc: linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org, Akira Yokosawa
 <akiyks@...il.com>
Subject: Re: [PATCH 12/12] docs: kdoc: Improve the output text accumulation

Em Thu, 10 Jul 2025 12:31:55 +0200
Mauro Carvalho Chehab <mchehab+huawei@...nel.org> escreveu:

> Em Thu, 10 Jul 2025 12:10:33 +0200
> Mauro Carvalho Chehab <mchehab+huawei@...nel.org> escreveu:
> 
> > From the above:
> > 
> > - It is not worth applying patch 12/12 as it makes the code slower;
> > - Python 3.13t (no-GIL version) had very bad results. It seems it
> >   still requires optimization;
> > - Python 3.9 is a lot worse (140% to 190%) when using list append;
> > - when there are not many concats, Python 3.13 is about 15% slower
> >   with lists than concat strings. It only approaches str concat
> >   when the number of concats is high.
> > 
> > With the above, clearly str += is faster than list append.
> > 
> > So, except if I did something wrong on this benchmark script, please
> > don't apply patch 12/12.  
> 
> And I did: I forgot the final line at the concat code to get the
> result as strings.
> 
> For explicit list:
> 	result = obj.output()
> 
> For implicit ones:
> 	result = str(obj)
> 
> Yet, the conclusion is similar. With Python 3.13:
> 
>     $ for i in python3.13; do for j in 1 10 100 1000; do $i /tmp/bench.py $((1000000/$j)) $j 1; done; done
>     1 strings in a loop with 1000000 interactions, repeating 24 times
>         str +=       : time: 41.42
>         list join    : time: 127.33: 207.42% slower than str +=
>     10 strings in a loop with 100000 interactions, repeating 24 times
>         str +=       : time: 27.15
>         list join    : time: 39.19: 44.36% slower than str +=
>     100 strings in a loop with 10000 interactions, repeating 24 times
>         str +=       : time: 24.84
>         list join    : time: 30.70: 23.57% slower than str +=
>     1000 strings in a loop with 1000 interactions, repeating 24 times
>         str +=       : time: 21.84
>         list join    : time: 27.85: 27.50% slower than str +=
> 
> Explict list concat was between ~30% to ~200% worse than str concat.

Looking for an explanation, PEP 509 and PEP 393 did Short String Optimization
and Inline Caching. This was applied Python 3.6, and Python 3.13 came with
extra string optimizations.

On the other hand, lists do more memory allocations, have some logic to
extend list growth and has an extra concat loop.

With that, contrary to popular belief, it sounds that str concat are
nowadays faster.

Thanks,
Mauro