linux-kernel - [PATCH v9 14/13] sphinx: parser_yaml.py: fix line numbers information

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4d1e0f5283ae1c6874cef272c5760035eb51278a.1752222934.git.mchehab+huawei@kernel.org>
Date: Fri, 11 Jul 2025 10:36:23 +0200
From: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To: Linux Doc Mailing List <linux-doc@...r.kernel.org>,
	Jonathan Corbet <corbet@....net>
Cc: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>,
	"Akira Yokosawa" <akiyks@...il.com>,
	"Breno Leitao" <leitao@...ian.org>,
	"David S. Miller" <davem@...emloft.net>,
	"Donald Hunter" <donald.hunter@...il.com>,
	"Eric Dumazet" <edumazet@...gle.com>,
	"Ignacio Encinas Rubio" <ignacio@...cinas.com>,
	"Jan Stancek" <jstancek@...hat.com>,
	"Marco Elver" <elver@...gle.com>,
	"Mauro Carvalho Chehab" <mchehab+huawei@...nel.org>,
	"Paolo Abeni" <pabeni@...hat.com>,
	"Randy Dunlap" <rdunlap@...radead.org>,
	"Ruben Wauters" <rubenru09@....com>,
	"Shuah Khan" <skhan@...uxfoundation.org>,
	joel@...lfernandes.org,
	linux-kernel-mentees@...ts.linux.dev,
	linux-kernel@...r.kernel.org,
	lkmm@...ts.linux.dev,
	netdev@...r.kernel.org,
	peterz@...radead.org,
	stern@...land.harvard.edu
Subject: [PATCH v9 14/13] sphinx: parser_yaml.py: fix line numbers information

As reported by Donald, this code:

	rst_parser = RSTParser()
	rst_parser.parse('\n'.join(result), document)

breaks line parsing. As an alternative, I tested a variant of it:

	rst_parser.parse(result, document)

but still line number was not preserved. As Donald noted,
standard Parser classes don't have a direct mechanism to preserve
line numbers from ViewList().

So, instead, let's use a mechanism similar to what we do already at
kerneldoc.py: call the statemachine mechanism directly there.

I double-checked when states and statemachine were introduced:
both were back in 2002. I also tested doc build with docutils 0.16
and 0.21.2. It worked with both, so it seems to be stable enough
for our needs.

Reported-by: Donald Hunter <donald.hunter@...il.com>
Closes: https://lore.kernel.org/linux-doc/m24ivk78ng.fsf@gmail.com/T/#u
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
---

PS.: I'm opting to send this as 14/13 to avoid respanning the entire
series again just due to this extra change.

 Documentation/sphinx/parser_yaml.py | 21 ++++++++++++++-------
 1 file changed, 14 insertions(+), 7 deletions(-)

diff --git a/Documentation/sphinx/parser_yaml.py b/Documentation/sphinx/parser_yaml.py
index 1602b31f448e..634d84a202fc 100755
--- a/Documentation/sphinx/parser_yaml.py
+++ b/Documentation/sphinx/parser_yaml.py
@@ -11,7 +11,9 @@ import sys
 
 from pprint import pformat
 
+from docutils import statemachine
 from docutils.parsers.rst import Parser as RSTParser
+from docutils.parsers.rst import states
 from docutils.statemachine import ViewList
 
 from sphinx.util import logging
@@ -56,6 +58,8 @@ class YamlParser(Parser):
 
     re_lineno = re.compile(r"\.\. LINENO ([0-9]+)$")
 
+    tab_width = 8
+
     def rst_parse(self, inputstring, document, msg):
         """
         Receives a ReST content that was previously converted by the
@@ -66,10 +70,18 @@ class YamlParser(Parser):
 
         result = ViewList()
 
+        self.statemachine = states.RSTStateMachine(state_classes=states.state_classes,
+                                                   initial_state='Body',
+                                                   debug=document.reporter.debug_flag)
+
         try:
             # Parse message with RSTParser
             lineoffset = 0;
-            for line in msg.split('\n'):
+
+            lines = statemachine.string2lines(msg, self.tab_width,
+                                              convert_whitespace=True)
+
+            for line in lines:
                 match = self.re_lineno.match(line)
                 if match:
                     lineoffset = int(match.group(1))
@@ -77,12 +89,7 @@ class YamlParser(Parser):
 
                 result.append(line, document.current_source, lineoffset)
 
-            # Fix backward compatibility with docutils < 0.17.1
-            if "tab_width" not in vars(document.settings):
-                document.settings.tab_width = 8
-
-            rst_parser = RSTParser()
-            rst_parser.parse('\n'.join(result), document)
+            self.statemachine.run(result, document)
 
         except Exception as e:
             document.reporter.error("YAML parsing error: %s" % pformat(e))
-- 
2.50.0