linux-ext4 - Re: [PATCH RFC v5 00/11] Ext4 Encoding and Case-insensitive support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190205181041.cdyt5jt7yrqswyy2@pali>
Date:   Tue, 5 Feb 2019 19:10:41 +0100
From:   Pali Rohár <pali.rohar@...il.com>
To:     Gabriel Krisman Bertazi <krisman@...labora.com>
Cc:     tytso@....edu, linux-fsdevel@...r.kernel.org,
        linux-ext4@...r.kernel.org, sfrench@...ba.org,
        darrick.wong@...cle.com, samba-technical@...ts.samba.org,
        jlayton@...nel.org, bfields@...ldses.org, paulus@...ba.org
Subject: Re: [PATCH RFC v5 00/11] Ext4 Encoding and Case-insensitive support

On Monday 28 January 2019 16:32:12 Gabriel Krisman Bertazi wrote:
> The main change presented here is a proposal to migrate the
> normalization method from NFKD to NFD.  After our discussions, and
> reviewing other operating systems and languages aspects, I am more
> convinced that canonical decomposition is more viable solution than
> compatibility decomposition, because it doesn't ignore eliminate any
> semantic meaning, like the definitive case of superscript numbers.  NFD
> is also the documented method used by HFS+ and APFS, so there is
> precedent. Notice however, that as far as my research goes, APFS doesn't
> completely follows NFD, and in some cases, like <compat> flags, it
> actually does NFKD, but not in others (<fraction>), where it applies the
> canonical form.  We take a more consistent approach and always do plain NFD.
> 
> This RFC, therefore, aims to resume/start conversation with some
> stalkeholders that may have something to say regarding the normalization
> method used.  I added people from SMB, NFS and FS development who
> might be interested on this.

Hello! I think that choice of NFD normalization is not right decision.
Some reasons:

1) NFD is not widely used. Even Apple does not use it (as you wrote
   Apple has own normalization form).

2) All filesystems which I known either do not use any normalization or
   use NFC.

3) Lot of existing Linux application generate file names in NFC.

4) Linux GUI libraries like Qt and Gtk generate strings from key strokes
   in NFC. So if user type file name in Qt/Gtk box it would be in NFC.

So why to use NFD in ext4 filesystem if Linux userspace ecosystem
already uses NFC?

NFD here just makes another layer of problems, unexpected things and
make it somehow different.

Why not rather choose NFS? It would be more compatible with Linux GUI
applications and also with Microsoft Windows systems, which uses NFC
too.

Please, really consider to not use NFD. Most Linux applications really
do not do any normalization or do NFC. And usage of decomposition form
for application which do not implement full Unicode grapheme algorithms
just make for them another problems.

Yes, there are still lot of legacy application which expect that one
code point = one visible symbol (therefore one Unicode grapheme). And
because GUI in most cases generates NFC strings, also existing file
names are in NFC, these application works in most cases without problem.
Force usage of NFD filenames just break them.

(PS: I think that only 2 programming languages implements Unicode
grapheme algorithms correctly: Elixir and Perl 6; which is not so much)

-- 
Pali Rohár
pali.rohar@...il.com

Download attachment "signature.asc" of type "application/pgp-signature" (196 bytes)