linux-kernel - Re: [PATCH v2] get_maintainer: correctly parse UTF-8 encoded names in files

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <f7510474a007223811da8b04efabf13376e53728.camel@perches.com>
Date:   Thu, 14 Dec 2023 07:57:54 -0800
From:   Joe Perches <joe@...ches.com>
To:     Alvin Šipraga <alvin@...s.dk>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Duje Mihanović <duje.mihanovic@...le.hr>,
        Konstantin Ryabitsev <konstantin@...uxfoundation.org>,
        linux-kernel@...r.kernel.org,
        Alvin Šipraga <alsi@...g-olufsen.dk>,
        Shawn Guo <shawnguo@...nel.org>,
        Andrew Morton <akpm@...ux-foundation.org>
Subject: Re: [PATCH v2] get_maintainer: correctly parse UTF-8 encoded names
 in files

On Thu, 2023-12-14 at 16:06 +0100, Alvin Šipraga wrote:
> From: Alvin Šipraga <alsi@...g-olufsen.dk>
> 
> While the script correctly extracts UTF-8 encoded names from the
> MAINTAINERS file, the regular expressions damage my name when parsing
> from .yaml files. Fix this by replacing the Latin-1-compatible regular
> expressions with the unicode property matcher \p{L}, which matches on
> any letter according to the Unicode General Category of letters.

OK

> It's also necessary to instruct Perl to open all files with UTF-8 encoding.

I doubt this.

> ---
> Changes in v2:
> - use '\p{L}' rather than '\p{Latin}', so that matching is even more
>   inclusive (i.e. match also Greek letters, CJK, etc.)
> - fix commit message to refer to tools mailing list, not b4 mailing list
> - Link to v1: https://lore.kernel.org/r/20231014-get-maintainers-utf8-v1-1-3af8c7aeb239@bang-olufsen.dk

OK

> diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
[]
> @@ -20,6 +20,7 @@ use Getopt::Long qw(:config no_auto_abbrev);
>  use Cwd;
>  use File::Find;
>  use File::Spec::Functions;
> +use open qw(:std :encoding(UTF-8));

I think this global use is unnecessary.


> @@ -442,7 +443,7 @@ sub maintainers_in_file {
>  	my $text = do { local($/) ; <$f> };
>  	close($f);
>  
> -	my @poss_addr = $text =~ m$[A-Za-zÀ-ÿ\"\' \,\.\+-]*\s*[\,]*\s*[\(\<\{]{0,1}[A-Za-z0-9_\.\+-]+\@[A-Za-z0-9\.-]+\.[A-Za-z0-9]+[\)\>\}]{0,1}$g;
> +	my @poss_addr = $text =~ m$[\p{L}\"\' \,\.\+-]*\s*[\,]*\s*[\(\<\{]{0,1}[A-Za-z0-9_\.\+-]+\@[A-Za-z0-9\.-]+\.[A-Za-z0-9]+[\)\>\}]{0,1}$g;
>  	push(@file_emails, clean_file_emails(@poss_addr));
>      }
>  }

Rather than open _all_ files in utf-8, perhaps the block
that opens a specific file to find maintainers

sub maintainers_in_file {
    my ($file) = @_;

    return if ($file =~ m@\bMAINTAINERS$@);

    if (-f $file && ($email_file_emails || $file =~ /\.yaml$/)) {
	open(my $f, '<', $file)
	    or die "$P: Can't open $file: $!\n";
	my $text = do { local($/) ; <$f> };
	close($f);
	...

should change the

	open(my $f...
to
	use open qw(:std :encoding(UTF-8));
	open(my $f...


And unrelated and secondarily, perhaps the
	$file =~ /\.yaml$/
test should be
	$file =~ /\.(?:yaml|dtsi?)$/

to also find any maintainer address in the dts* files

https://lore.kernel.org/lkml/20231028174656.GA3310672@bill-the-cat/T/