linux-kernel - Re: why do we need utf8 normalization when compare name?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 3 Mar 2020 18:13:56 +0800
From:   lampahome <pahome.chen@...lab.org>
To:     Aleksa Sarai <cyphar@...har.com>
Cc:     linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: why do we need utf8 normalization when compare name?

> Unicode normalisation will take the strings "ñ" (U+00F1) and "n◌̃"
> (U+006E U+0303) and turn them into the same Unicode string. Note that
> there are four kinds of Unicode normalisation (NFD, NFC, NFKD, NFKC), so
> what precise string you end up with depends on which form you're using.
> Linux uses NFD, I believe.


> And yes, once the strings are normalised and encoded as UTF-8 you then
> do a byte-by-byte comparison (if the comparison is case-insensitive then
> fs/unicode/... will case-fold the Unicode symbols during normalisation).
>

What I'm confused is why encoded as utf-8 after normalize finished?
>From above, turn "ñ" (U+00F1) and "n◌̃" (U+006E U+0303) into the same
Unicode string. Then why should we just compare bytes from normalized.