linux-kernel - Re: [PATCH 5/5] MIPS: LLVMLinux: Silence unicode warnings when preprocessing assembly.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <yw1xh9v0a55p.fsf@unicorn.mansr.com>
Date:	Thu, 05 Feb 2015 12:56:02 +0000
From:	Måns Rullgård <mans@...sr.com>
To:	"Maciej W. Rozycki" <macro@...ux-mips.org>
Cc:	Toma Tabacu <Toma.Tabacu@...tec.com>,
	Daniel Sanders <Daniel.Sanders@...tec.com>,
	Ralf Baechle <ralf@...ux-mips.org>,
	Paul Burton <Paul.Burton@...tec.com>,
	Paul Bolle <pebolle@...cali.nl>,
	"Steven J. Hill" <Steven.Hill@...tec.com>,
	Manuel Lauss <manuel.lauss@...il.com>,
	Jim Quinlan <jim2101024@...il.com>,
	"linux-mips\@linux-mips.org" <linux-mips@...ux-mips.org>,
	"linux-kernel\@vger.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH 5/5] MIPS: LLVMLinux: Silence unicode warnings when preprocessing assembly.

"Maciej W. Rozycki" <macro@...ux-mips.org> writes:

> On Thu, 5 Feb 2015, Toma Tabacu wrote:
>
>> > 2. It considers these character pairs to be unicode escapes in the first 
>> >    place given that they do not follow the syntax required for such 
>> >    escapes, that is `\unnnn', where `n' are hex digits.
>> > 
>> 
>> It doesn't actually treat them as unicode escapes, but it still warns
>> the user, in case they were meant to be unicode escapes. Here's the
>> warning message:
>> 
>> arch/mips/include/asm/asmmacro.h:197:51: warning: \u used with no following hex digits; treating as '\' followed by identifier [-Wunicode]
>>          .word  0x41000000 | (\rt << 16) | (\rd << 11) | (\u << 5) | (\sel)
>>                                                           ^
>> I'll add it to the summary in v2.
>
>  Thanks, that makes things clearer.  It always makes sense to include the 
> exact error message produced where applicable or otherwise people do not 
> necessarily know what the matter is.
>
>> > Of course it may be reasonable for us to work this bug around as we've 
>> > been doing for years with GCC, but has the issue been reported back to 
>> > clang maintainers?  What was their response?
>> > 
>> 
>> It hasn't been reported, but I don't think they would agree with removing
>> unicode escape sequences from the assembler-with-cpp mode because it is
>> currently being used for other languages as well, not just assembly.
>
>  First, preprocessing rules surely have to be language specific.  The C 
> language standard does not specify what the preprocessor is meant to do 
> (if anything) for other languages.  GCC or clang -- that's no different.  
>
>  The assembly language has a different syntax and `\u' has a different 
> meaning in the context of assembly macro expansion than it would have in a 
> name of a symbol, where such a Unicode escape sequence might indeed be 
> interpreted as such and character encoded propagated to the symbol 
> produced.  But that's up to the assembler -- GAS for example does not 
> AFAIK support Unicode escape sequences in symbol names right now, but I 
> suppose such a feature could be added if desired.
>
>  Which prompts another question of course: how does the clang C compiler 
> represent Unicode characters in identifiers in its assembly output?
>
>  I have looked into the C language standard and it appears to me like the 
> translation phase to interpret universal character names at has not been 
> defined.  This is probably why the standard does specify the result of 
> pasting preprocessor tokens together as undefined if a universal character 
> name is produced this way.

That is my interpretation as well.

>  Consequently I think an important question in this context is: does 
> clang's preprocessor actually convert these sequences anyhow before 
> passing them down to the compiler?  How for example does C output from a 
> trivial example that contains such Unicode escape sequences look like 
> then?
>
>> One such language is Haskell (ghc, to be more specific), for which
>> the clang developers had to actually stop the preprocessor from
>> enforcing the C universal character name restrictions in
>> assembler-with-cpp mode, which suggests that ghc wants the
>> preprocessor to check for unicode escape sequences.
>> 
>> At the moment, we can either disable -Wunicode for asmmacro.h or
>> refrain from using '\u' as an identifier.
>
>  To be clear: it's `u' here that is the identifier, the leading `\' is 
> merely how assembly syntax has been specified for references to macro 
> arguments.  And TBH I find banning any macro arguments starting with `u' 
> rather silly.

Agreed.

> I'm leaning towards considering having -Wunicode disabled for all
> assembly sources, or maybe even for the whole Linux compilation, the
> right solution.  It's not like we have a need for Unicode identifiers.

It might be an idea to disable -Wunicode and have checkpatch warn about
Unicode escapes instead if people are worried about this.  Personally, I
doubt there's much cause for concern here.

-- 
Måns Rullgård
mans@...sr.com
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/