linux-kernel - Re: [PATCH v2] Convert properly UTF-8 to UTF-16

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <5073AB7A.3020103@suse.com>
Date:	Tue, 09 Oct 2012 10:13:38 +0530
From:	Suresh Jayaraman <sjayaraman@...e.com>
To:	Frediano Ziglio <frediano.ziglio@...rix.com>
Cc:	"smfrench@...il.com" <smfrench@...il.com>,
	"sfrench@...ba.org" <sfrench@...ba.org>,
	"jlayton@...hat.com" <jlayton@...hat.com>,
	"linux-cifs@...r.kernel.org" <linux-cifs@...r.kernel.org>,
	"samba-technical@...ts.samba.org" <samba-technical@...ts.samba.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v2] Convert properly UTF-8 to UTF-16

On 10/08/2012 01:48 PM, Frediano Ziglio wrote:
> On Wed, 2012-10-03 at 14:49 -0500, Steve French wrote:
>> Merged - but doesn't the reverse also have to be added in cifs_from_utf16?  ie
>>
>>           utf16s_to_utf8s(uni, ... );
>>
> 
> Not strictly necessary, at least to be able to mount shares.
> 
>> I am glad that someone added these multiword handling routines into
>> the kernel for FAT - this has been something we have wanted for a long
>> time in cifs (and smb2/smb3).  Note the comment in
>> fs/cifs/cifs_unicode.c
>>
>> / * Note that some windows versions actually send multiword UTF-16 characters
>>  * instead of straight UTF16-2. The linux nls routines however aren't able to
>>  * deal with those characters properly. In the event that we get some of
>>  * those characters, they won't be translated properly.
>>  */
>> int
>> cifs_from_utf16(char *to, const __le16 *from, int tolen, int fromlen,
>>                  const struct nls_table *codepage, bool mapchar)
>>
> 
> Should not be UCS-2 instead of UTF16-2 ??
> 
>>
>> We could really use some nls test cases for cifs/smb2/smb3/nfs4 which
>> basically did various file, directory, symlink create/rename/delete
>> operations with various hard to map characters so we can test copying
>> to and from the server and ensure that we get the name mappings right
>> for these (and don't ever regress).   Fortunately smb2/smb3 is only
>> unicode so we don't have to deal with mappings to other codepages from
>> utf8
>>
> 
> Do you have some framework/hook to put these tests ?
> 

I recently wrote cifstests to primarily provide a basic infrastructure
for adding regression tests for cifs. It's written in python and the
plan to be able to use python or C bindings for python. You might
consider adding tests to it.

   https://github.com/sureshjayaram/cifstests


Thanks
Suresh
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/