linux-kernel - Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20191001113506.12720205@lwn.net>
Date:   Tue, 1 Oct 2019 11:35:06 -0600
From:   Jonathan Corbet <corbet@....net>
To:     Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
Cc:     Kees Cook <keescook@...omium.org>,
        Jani Nikula <jani.nikula@...ux.intel.com>,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCH 0/2] docs: Programmatically render MAINTAINERS into ReST

On Tue, 1 Oct 2019 12:09:30 -0300
Mauro Carvalho Chehab <mchehab+samsung@...nel.org> wrote:

> >   Sphinx parallel build error:
> >   UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 8: ordinal not in range(128)
> > 
> > For extra fun, the build process simply hangs, requiring a ^C to blow it
> > away.  You've managed to get new behavior out of Sphinx that I've not seen
> > before, congratulations :)
> > 
> > This almost certainly has to do with the fact that I'm (intentionally)
> > running the Python2 Sphinx build here.  Something's not doing unicode that
> > should be.
> > 
> > I would suggest that we might just want to repair this before merging this
> > feature.  Either that, or we bite the bullet and deprecate the use of
> > Python 2 entirely - something that's probably not too far into our future
> > regardless.  Anybody have thoughts on that matter?  
> 
> I'm almost sure we got this already. If I'm not mistaken, the solution
> is to add the encoding line after shebang.

As mentioned before, that's not it.  The problem is that we're feeding
UTF8 into Sphinx as an ordinary str, then sphinx is trying to decode it
as ASCII. The attached hack makes things work.

Kees, I can either just keep this fix (breaking bisectability of the docs
build :) or you can send a new version - up to you.

Thanks,

jon

>From c32bfbb07a7840662ba3b367c61b7f2946028b27 Mon Sep 17 00:00:00 2001
From: Jonathan Corbet <corbet@....net>
Date: Tue, 1 Oct 2019 11:26:20 -0600
Subject: [PATCH] docs: make maintainers_include work with Python 2

The MAINTAINERS file contains UTF-8 data, so Python 2 code has to be
explicit about moving it into the Unicode realm.  Explicitly decode all
data from the file as soon as we read it.

This hack can go away once we deprecate Python 2 support.

Signed-off-by: Jonathan Corbet <corbet@....net>
---
 Documentation/sphinx/maintainers_include.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/Documentation/sphinx/maintainers_include.py b/Documentation/sphinx/maintainers_include.py
index de62dd3dabd5..705ba9fd5336 100755
--- a/Documentation/sphinx/maintainers_include.py
+++ b/Documentation/sphinx/maintainers_include.py
@@ -19,6 +19,7 @@ u"""
 
 import re
 import os.path
+import sys
 
 from docutils import statemachine
 from docutils.utils.error_reporting import ErrorString
@@ -60,6 +61,8 @@ class MaintainersInclude(Include):
         field_content = ""
 
         for line in open(path):
+            if sys.version_info.major == 2:
+                line = unicode(line, 'utf-8')
             # Have we reached the end of the preformatted Descriptions text?
             if descriptions and line.startswith('Maintainers'):
                 descriptions = False
-- 
2.21.0