linux-kernel - Re: why do we need utf8 normalization when compare name?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200302103754.nsvtne2vvduug77e@yavin>
Date:   Mon, 2 Mar 2020 21:37:54 +1100
From:   Aleksa Sarai <cyphar@...har.com>
To:     lampahome <pahome.chen@...lab.org>
Cc:     linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: why do we need utf8 normalization when compare name?

On 2020-03-02, lampahome <pahome.chen@...lab.org> wrote:
> According to case insensitive since kernel 5.2, d_compare will
> transform string into normalized form and then compare.
>
> But why do we need this normalization function? Could we just compare
> by utf8 string?

The problem is that there are multiple ways to represent the same glyph
in Unicode -- for instance, you can represent Å (the symbol for
angstrom) as both U+212B and U+0041 U+030A (the latin letter "A"
followed by the ring-above symbol "°"). Different software may choose to
represent the same glyphs in different Unicode forms, hence the need for
normalisation.

[1] is the Wikipedia article that describes this problem and what the
different kinds of Unicode normalisation are.

[1]: https://en.wikipedia.org/wiki/Unicode_equivalence

-- 
Aleksa Sarai
Senior Software Engineer (Containers)
SUSE Linux GmbH
<https://www.cyphar.com/>

Download attachment "signature.asc" of type "application/pgp-signature" (229 bytes)