linux-kernel - Re: [ANNOUNCE] Kconfiglib menuconfig implementation

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAFkk2KQX=j5_VCSRR1fm31x=yC3zse47fnpJaqpLGOLw-v5QkA@mail.gmail.com>
Date:   Mon, 14 May 2018 20:12:21 +0200
From:   Ulf Magnusson <ulfalizer@...il.com>
To:     Pavel Machek <pavel@....cz>
Cc:     Randy Dunlap <rdunlap@...radead.org>,
        Linux Kbuild mailing list <linux-kbuild@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [ANNOUNCE] Kconfiglib menuconfig implementation

On Mon, May 14, 2018 at 6:58 PM, Pavel Machek <pavel@....cz> wrote:
> Hi!
>
>> > config CAN_PEAK_PCIEFD
>> >         depends on PCI
>> >         tristate "PEAK-System PCAN-PCIe FD cards"
>> >         ---help---
>> >           This driver adds support for the PEAK-System PCI Express FD
>> >           CAN-FD cards family.
>> >           These 1x or 2x CAN-FD channels cards offer CAN 2.0 a/b as well as
>> >           CAN-FD access to the CAN bus. Besides the nominal bitrate of up to
>> >           1 Mbit/s, the data bytes of CAN-FD frames can be transmitted with
>> >           up to 12 Mbit/s. A galvanic isolation of the CAN ports protects the
>> >           electronics of the card and the respective computer against
>> >           disturbances of up to 500 Volts. The PCAN-PCI Express FD can be
>> >           operated with ambient temperatures in a range of -40 to +85 °C.
>> >
>> >
>> > kconfiglib.KconfigSyntaxError:
>> > Malformed ascii in drivers/net/can/peak_canfd/Kconfig
>> > Context: b't temperatures in a range of -40 to +85 \xc2\xb0C.\n'
>> > Problematic data: b'\xc2'
>> > Reason: ordinal not in range(128)
>> >
>> >
>> > thanks,
>> > --
>> > ~Randy
>>
>> Thanks for trying it out!
>>
>> You're probably running in the C locale, which implies an ASCII
>> encoding. That has caused enough trouble that the Python devs decided
>> to automatically convert it to UTF-8 in Python 3.7:
>> https://www.python.org/dev/peps/pep-0538/. LC_CTYPE=C.UTF-8 should fix
>> it.
>>
>> It's a bit silly to have it crash for something like that though. I
>> could force UTF-8 instead of respecting the locale (though it feels
>> neater to respect settings), or tell Python to ignore decoding errors.
>> Should probably do something at least...
>
> Well, Kconfig files are utf-8, as is the rest of kernel.

Yeah, it's the only sane assumption anyway.

>
> So yes, you should respect $LC_CTYPE for display etc, but you should always
> assume Kconfig files are utf-8... and that does not need to be optional.

Already made it optional, with UTF-8 as the default (via
Kconfig.__init__(..., encoding="utf-8")).

I did a similar thing to what PEP 538 does for the menuconfig:
https://github.com/ulfalizer/Kconfiglib/blob/master/menuconfig.py#L2271.
The C locale is converted to a UTF-8 locale for LC_CTYPE if possible.
Other locales are respected. That'll be the default behavior in Python
3.7, and prevents issues with get_wch().

>
>                                                                         Pavel

Cheers,
Ulf