linux-kernel - Re: Unification of filesystem encoding options

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200107200301.GE3619@mit.edu>
Date:   Tue, 7 Jan 2020 15:03:01 -0500
From:   "Theodore Y. Ts'o" <tytso@....edu>
To:     Pali Rohár <pali.rohar@...il.com>
Cc:     Jan Kara <jack@...e.cz>, linux-kernel@...r.kernel.org,
        linux-fsdevel@...r.kernel.org,
        linux-ntfs-dev@...ts.sourceforge.net, linux-cifs@...r.kernel.org,
        Alexander Viro <viro@...iv.linux.org.uk>,
        Luis de Bethencourt <luisbg@...nel.org>,
        Salah Triki <salah.triki@...il.com>,
        Steve French <sfrench@...ba.org>,
        OGAWA Hirofumi <hirofumi@...l.parknet.co.jp>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        David Sterba <dsterba@...e.com>,
        Dave Kleikamp <shaggy@...nel.org>,
        Anton Altaparmakov <anton@...era.com>,
        Jan Kara <jack@...e.com>, Eric Sandeen <sandeen@...hat.com>,
        Namjae Jeon <linkinjeon@...il.com>,
        Pavel Machek <pavel@....cz>,
        Christoph Hellwig <hch@...radead.org>
Subject: Re: Unification of filesystem encoding options

On Tue, Jan 07, 2020 at 06:38:42PM +0100, Pali Rohár wrote:
> Adding support for case-insensitivity into UTF-8 NLS encoding would mean
> to create completely new kernel NLS API (which would support variable
> length encodings) and rewrite all NLS filesystems to use this new API.
> Also all existing NLS encodings would be needed to port into this new
> API.
> 
> It is really something which have a value? Just because of UTF-8?
> 
> For me it looks like better option would be to remove UTF-8 NLS encoding
> as it is broken. Some filesystems already do not use NLS API for their
> UTF-8 support (e.g. vfat, udf or newly prepared exfat). And others could
> be modified/extended/fixed in similar way.

You didn't mention ext4 and f2fs, which is using the Unicode code in
fs/unicode for its case-folding and normalization support.  Ext4 and
f2fs only supports utf-8, so using the NLS API would have added no
value --- and it as you pointed out, the NLS API doesn't support
variable length encoding anyway.  In contrast the fs/unicode functions
have support for full Unicode case folding and normalization, and
currently has the latest Unicode 12.1 tables (released May 2019).

What I'd suggest is to create a new API, enhancing the functions in
fs/unicode, to support those file systems that need to deal with
UTF-16 and UTF-32 for their on-disk directory format, and that we
assume that for the most part, userspace *will* be using a UTF-8
encoding for the user<->kernel interface.  We can keep the existing
NLS interface and mount options for legacy support, but in my opinion
it's not worth the effort to try to do anything else.

					- Ted