linux-kernel - [PATCH 4.19 10/44] scripts/spdxcheck.py: always open files in binary mode

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20181218163928.707844067@linuxfoundation.org>
Date:   Tue, 18 Dec 2018 17:39:22 +0100
From:   Greg Kroah-Hartman <gregkh@...uxfoundation.org>
To:     linux-kernel@...r.kernel.org
Cc:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        stable@...r.kernel.org, Thierry Reding <treding@...dia.com>,
        Jeremy Cline <jcline@...hat.com>,
        Thomas Gleixner <tglx@...utronix.de>,
        Jonathan Corbet <corbet@....net>,
        Joe Perches <joe@...ches.com>,
        Uwe Kleine-König 
        <u.kleine-koenig@...gutronix.de>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Linus Torvalds <torvalds@...ux-foundation.org>
Subject: [PATCH 4.19 10/44] scripts/spdxcheck.py: always open files in binary mode

4.19-stable review patch.  If anyone has any objections, please let me know.

------------------

From: Thierry Reding <treding@...dia.com>

commit 3a6ab5c7dc114057fd67750e308e1745dafc0e6a upstream.

The spdxcheck script currently falls over when confronted with a binary
file (such as Documentation/logo.gif).  To avoid that, always open files
in binary mode and decode line-by-line, ignoring encoding errors.

One tricky case is when piping data into the script and reading it from
standard input.  By default, standard input will be opened in text mode,
so we need to reopen it in binary mode.

The breakage only happens with python3 and results in a
UnicodeDecodeError (according to Uwe).

Link: http://lkml.kernel.org/r/20181212131210.28024-1-thierry.reding@gmail.com
Fixes: 6f4d29df66ac ("scripts/spdxcheck.py: make python3 compliant")
Signed-off-by: Thierry Reding <treding@...dia.com>
Reviewed-by: Jeremy Cline <jcline@...hat.com>
Cc: Thomas Gleixner <tglx@...utronix.de>
Cc: Jonathan Corbet <corbet@....net>
Cc: Joe Perches <joe@...ches.com>
Cc: Uwe Kleine-König <u.kleine-koenig@...gutronix.de>
Cc: <stable@...r.kernel.org>
Signed-off-by: Andrew Morton <akpm@...ux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@...ux-foundation.org>
Signed-off-by: Greg Kroah-Hartman <gregkh@...uxfoundation.org>

---
 scripts/spdxcheck.py |    6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

--- a/scripts/spdxcheck.py
+++ b/scripts/spdxcheck.py
@@ -168,6 +168,7 @@ class id_parser(object):
         self.curline = 0
         try:
             for line in fd:
+                line = line.decode(locale.getpreferredencoding(False), errors='ignore')
                 self.curline += 1
                 if self.curline > maxlines:
                     break
@@ -249,12 +250,13 @@ if __name__ == '__main__':
 
     try:
         if len(args.path) and args.path[0] == '-':
-            parser.parse_lines(sys.stdin, args.maxlines, '-')
+            stdin = os.fdopen(sys.stdin.fileno(), 'rb')
+            parser.parse_lines(stdin, args.maxlines, '-')
         else:
             if args.path:
                 for p in args.path:
                     if os.path.isfile(p):
-                        parser.parse_lines(open(p), args.maxlines, p)
+                        parser.parse_lines(open(p, 'rb'), args.maxlines, p)
                     elif os.path.isdir(p):
                         scan_git_subtree(repo.head.reference.commit.tree, p)
                     else: