linux-kernel - RE: [PATCH 5/5] MIPS: LLVMLinux: Silence unicode warnings when preprocessing assembly.

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <alpine.LFD.2.11.1502051107150.22715@eddie.linux-mips.org>
Date:	Thu, 5 Feb 2015 12:35:42 +0000 (GMT)
From:	"Maciej W. Rozycki" <macro@...ux-mips.org>
To:	Toma Tabacu <Toma.Tabacu@...tec.com>
cc:	Daniel Sanders <Daniel.Sanders@...tec.com>,
	Ralf Baechle <ralf@...ux-mips.org>,
	Paul Burton <Paul.Burton@...tec.com>,
	Paul Bolle <pebolle@...cali.nl>,
	"Steven J. Hill" <Steven.Hill@...tec.com>,
	Manuel Lauss <manuel.lauss@...il.com>,
	Jim Quinlan <jim2101024@...il.com>,
	"linux-mips@...ux-mips.org" <linux-mips@...ux-mips.org>,
	"linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: RE: [PATCH 5/5] MIPS: LLVMLinux: Silence unicode warnings when
 preprocessing assembly.

On Thu, 5 Feb 2015, Toma Tabacu wrote:

> > 2. It considers these character pairs to be unicode escapes in the first 
> >    place given that they do not follow the syntax required for such 
> >    escapes, that is `\unnnn', where `n' are hex digits.
> > 
> 
> It doesn't actually treat them as unicode escapes, but it still warns the user,
> in case they were meant to be unicode escapes. Here's the warning message:
> 
> arch/mips/include/asm/asmmacro.h:197:51: warning: \u used with no following hex digits; treating as '\' followed by identifier [-Wunicode]
>          .word  0x41000000 | (\rt << 16) | (\rd << 11) | (\u << 5) | (\sel)
>                                                           ^
> I'll add it to the summary in v2.

 Thanks, that makes things clearer.  It always makes sense to include the 
exact error message produced where applicable or otherwise people do not 
necessarily know what the matter is.

> > Of course it may be reasonable for us to work this bug around as we've 
> > been doing for years with GCC, but has the issue been reported back to 
> > clang maintainers?  What was their response?
> > 
> 
> It hasn't been reported, but I don't think they would agree with removing
> unicode escape sequences from the assembler-with-cpp mode because it is
> currently being used for other languages as well, not just assembly.

 First, preprocessing rules surely have to be language specific.  The C 
language standard does not specify what the preprocessor is meant to do 
(if anything) for other languages.  GCC or clang -- that's no different.  

 The assembly language has a different syntax and `\u' has a different 
meaning in the context of assembly macro expansion than it would have in a 
name of a symbol, where such a Unicode escape sequence might indeed be 
interpreted as such and character encoded propagated to the symbol 
produced.  But that's up to the assembler -- GAS for example does not 
AFAIK support Unicode escape sequences in symbol names right now, but I 
suppose such a feature could be added if desired.

 Which prompts another question of course: how does the clang C compiler 
represent Unicode characters in identifiers in its assembly output?

 I have looked into the C language standard and it appears to me like the 
translation phase to interpret universal character names at has not been 
defined.  This is probably why the standard does specify the result of 
pasting preprocessor tokens together as undefined if a universal character 
name is produced this way.

 Consequently I think an important question in this context is: does 
clang's preprocessor actually convert these sequences anyhow before 
passing them down to the compiler?  How for example does C output from a 
trivial example that contains such Unicode escape sequences look like 
then?

> One such language is Haskell (ghc, to be more specific), for which the clang
> developers had to actually stop the preprocessor from enforcing the C universal
> character name restrictions in assembler-with-cpp mode, which suggests that ghc
> wants the preprocessor to check for unicode escape sequences.
> 
> At the moment, we can either disable -Wunicode for asmmacro.h or refrain from
> using '\u' as an identifier.

 To be clear: it's `u' here that is the identifier, the leading `\' is 
merely how assembly syntax has been specified for references to macro 
arguments.  And TBH I find banning any macro arguments starting with `u' 
rather silly.  I'm leaning towards considering having -Wunicode disabled 
for all assembly sources, or maybe even for the whole Linux compilation, 
the right solution.  It's not like we have a need for Unicode identifiers.  

 What's the exact semantics of -Wunicode for clang?

  Maciej
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/