linux-kernel - Re: vfat: Broken case-insensitive support for UTF-8

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 20 Jan 2020 16:20:09 +0100
From:   Pali Rohár <pali.rohar@...il.com>
To:     David Laight <David.Laight@...LAB.COM>
Cc:     OGAWA Hirofumi <hirofumi@...l.parknet.co.jp>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>,
        "linux-fsdevel@...r.kernel.org" <linux-fsdevel@...r.kernel.org>,
        "Theodore Y. Ts'o" <tytso@....edu>,
        Namjae Jeon <linkinjeon@...il.com>,
        Gabriel Krisman Bertazi <krisman@...labora.com>
Subject: Re: vfat: Broken case-insensitive support for UTF-8

On Monday 20 January 2020 15:07:20 David Laight wrote:
> From: Pali Rohár
> > Sent: 20 January 2020 11:05
> > On Monday 20 January 2020 13:04:42 OGAWA Hirofumi wrote:
> > > Pali Rohár <pali.rohar@...il.com> writes:
> > >
> > > > Which means that fat_name_match(), vfat_hashi() and vfat_cmpi() are
> > > > broken for vfat in UTF-8 mode.
> > >
> > > Right. It is a known issue.
> > 
> > Could be this issue better documented? E.g. in mount(8) manpage where
> > are written mount options for vfat? I think that people should be aware
> > of this issue when they use "utf8=1" mount option.
> 
> What happens if the filesystem has filenames that invalid UTF8 sequences

Could you please describe what you mean by this question?

VFAT filesystem stores file names in UTF-16. Therefore you cannot have
UTF-8 on FS (and therefore also you cannot have invalid UTF-8).

Ehm... UTF-16 is not fully truth, MS FAT32 implementations allows half
of UTF-16 surrogate pair stored in FS.

Therefore practically, on VFAT you can store any uint16_t[] sequence as
filename, there is no invalid sequence (except those characters like
:<>?... which are invalid in MS-DOS).

If by "the filesystem has filenames" you do not mean filesystem file
names, but rather Linux VFS file names (e.g. you call creat() call with
invalid UTF-8 sequence) then function utf8s_to_utf16s() (called in
namei_vfat.c) fails and returns error. Which should be propagated to
open() / creat() call that it is not possible to create filename with
such UTF-8 sequence.

> or multiple filenames that decode from UTF8 to the same 'wchar' value.

This is not possible. There is 1:1 mapping between UTF-8 sequence and
Unicode code point. wchar_t in kernel represent either one Unicode code
point (limited up to U+FFFF in NLS framework functions) or 2bytes in
UTF-16 sequence (only in utf8s_to_utf16s() and utf16s_to_utf8s()
functions).

> Never mind ones that are just case-differences for the same filename.
> 
> UTF8 is just so broken it should never have been allowed to become
> a standard.

Well, UTF-16 is worse then UTF-8... incompatible with ASCII, variable
length and space consuming.

-- 
Pali Rohár
pali.rohar@...il.com