linux-kernel - [PATCH v2 38/38] doc/sphinx: Track line-number of starting blocks

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <d90368f2fa7ded7c56d214aef087e88bba5199e7.1465031816.git.jani.nikula@intel.com>
Date:	Sat,  4 Jun 2016 14:37:39 +0300
From:	Jani Nikula <jani.nikula@...el.com>
To:	Jonathan Corbet <corbet@....net>
Cc:	Jani Nikula <jani.nikula@...el.com>,
	Markus Heiser <markus.heiser@...marit.de>,
	Daniel Vetter <daniel.vetter@...ll.ch>,
	Grant Likely <grant.likely@...retlab.ca>,
	Mauro Carvalho Chehab <mchehab@....samsung.com>,
	Keith Packard <keithp@...thp.com>,
	LKML <linux-kernel@...r.kernel.org>, linux-doc@...r.kernel.org,
	Hans Verkuil <hverkuil@...all.nl>
Subject: [PATCH v2 38/38] doc/sphinx: Track line-number of starting blocks

From: Daniel Vetter <daniel.vetter@...ll.ch>

Design is pretty simple: kernel-doc inserts breadcrumbs with line
numbers, and sphinx picks them up. At first I went with a sphinx
comment, but inserting those at random places seriously upsets the
parser, and must be filtered. Hence why this version now uses "#define
LINEO " since one of these ever escape into output it's pretty clear
there is a bug.

It seems to work well, and at least the 2-3 errors where sphinx
complained about something that was not correct in kernel-doc text the
line numbers matched up perfectly.

v2: Instead of noodling around in the parser state machine, create
a ViewList and parse it ourselves. This seems to be the recommended
way, per Jani's suggestion.

v3:
- Split out ViewList pach. Splitting the kernel-doc changes from the
  sphinx ones isn't possible, since emitting the LINENO lines wreaks
  havoc with the rst formatting. We must filter them.

- Improve the regex per Jani's suggestions, and compile it just once
  for speed.

- Now that LINENO lines are eaten, also add them to function parameter
  descriptions. Much less content and offset than for in-line struct
  member descriptions, but still nice to know which exact continuation
  line upsets sphinx.

- Simplify/clarify the line +/-1 business a bit.

v4: Split out the scripts/kernel-doc changes and make line-numbers
opt-in, as suggested by Jani.

Cc: Jani Nikula <jani.nikula@...el.com>
Cc: linux-doc@...r.kernel.org
Cc: Jonathan Corbet <corbet@....net>
Signed-off-by: Daniel Vetter <daniel.vetter@...ll.ch>
Signed-off-by: Jani Nikula <jani.nikula@...el.com>
---
 Documentation/sphinx/kernel-doc.py | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)

diff --git a/Documentation/sphinx/kernel-doc.py b/Documentation/sphinx/kernel-doc.py
index bd422870101e..4adfb0e91ecc 100644
--- a/Documentation/sphinx/kernel-doc.py
+++ b/Documentation/sphinx/kernel-doc.py
@@ -30,6 +30,7 @@
 import os
 import subprocess
 import sys
+import re
 
 from docutils import nodes, statemachine
 from docutils.statemachine import ViewList
@@ -50,7 +51,7 @@ class KernelDocDirective(Directive):
 
     def run(self):
         env = self.state.document.settings.env
-        cmd = [env.config.kerneldoc_bin, '-rst']
+        cmd = [env.config.kerneldoc_bin, '-rst', '-enable-lineno']
 
         filename = env.config.kerneldoc_srctree + '/' + self.arguments[0]
 
@@ -93,7 +94,19 @@ class KernelDocDirective(Directive):
                 sys.stderr.write(err)
 
             lines = statemachine.string2lines(out, tab_width, convert_whitespace=True)
-            result = ViewList(lines, source)
+            result = ViewList()
+
+            lineoffset = 0;
+            line_regex = re.compile("^#define LINENO ([0-9]+)$")
+            for line in lines:
+                match = line_regex.search(line)
+                if match:
+                    # sphinx counts lines from 0
+                    lineoffset = int(match.group(1)) - 1
+                    # we must eat our comments since the upset the markup
+                else:
+                    result.append(line, source, lineoffset)
+                    lineoffset += 1
 
             node = nodes.section()
             node.document = self.state.document
-- 
2.1.4