linux-kernel - Re: OT: character encodings (was: Linux 2.6.20-rc4)

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20070107191730.GD21133@flint.arm.linux.org.uk>
Date:	Sun, 7 Jan 2007 19:17:30 +0000
From:	Russell King <rmk+lkml@....linux.org.uk>
To:	Alan <alan@...rguk.ukuu.org.uk>
Cc:	David Woodhouse <dwmw2@...radead.org>,
	Tilman Schmidt <tilman@...p.cc>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: OT: character encodings (was: Linux 2.6.20-rc4)

On Sun, Jan 07, 2007 at 06:21:51PM +0000, Alan wrote:
> > So, in short, UTF-8 is all fine and dandy if your _entire_ universe
> > is UTF-8 enabled.  If you're operating in a mixed charset environment
> > it's one bloody big pain in the butt.
> 
> Net ASCII is 7bit and is 1:1 mapped with UTF-8 unicode.

The same is true of ISO-8859-1.

> It's just old broken 8bit encodings that are problematic.
> 
> The kernel maintainers/help/config pretty consistently use UTF8

As I've tried to point out, that's not universally true.  For instance:

commit 24ebead82bbf9785909d4cf205e2df5e9ff7da32
tree 921f686860e918a01c3d3fb6cd106ba82bf4ace6
parent 264166e604a7e14c278e31cadd1afb06a7d51a11
author Rafa³ Bilski <rafalbilski@...eria.pl> 1167691774 +0100
committer Dave Jones <davej@...hat.com> 1167799119 -0500

and looking at that "author" closer with od:

0000140 74 68 6f 72 20 52 61 66 61 b3 20 42 69 6c 73 6b
          t   h   o   r       R   a   f   a   ³       B   i   l   s   k

clearly not UTF-8.  I doubt whether any of the commits I do on my
en_GB ISO-8859-1 systems end up being UTF-8 encoded.

And _this_ is the problem when it comes to generating the logs,
irrespective of whether or not Linus loads UTF-8 data into an
ISO-8859-1 message.  For all we know, Linus' system could be using
an ISO-8859 charset rather than UTF-8.

But the point is there is charset damage which has happened _long_ before
Linus' action.  There is no character set defined for the contents of git
repositories, and as such the output of the git tools can not be
interpreted as any one single character set.

All that UTF-8 has done is added to the "which charset is this data"
problem rather than actually solving any proper real life problem.

-- 
Russell King
 Linux kernel    2.6 ARM Linux   - http://www.arm.linux.org.uk/
 maintainer of:
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/