linux-kernel - Re: [PATCH v1 12/22] docs: driver-api: add .rst files from the main dir

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [day] [month] [year] [list]

Message-ID: <20190619120229.78ce6e07@coco.lan>
Date:   Wed, 19 Jun 2019 12:02:29 -0300
From:   Mauro Carvalho Chehab <mchehab+samsung@...nel.org>
To:     Jonathan Corbet <corbet@....net>
Cc:     Peter Zijlstra <peterz@...radead.org>,
        Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH v1 12/22] docs: driver-api: add .rst files from the main
 dir

Hi Jon,

Em Wed, 19 Jun 2019 08:13:53 -0600
Jonathan Corbet <corbet@....net> escreveu:

> On Wed, 19 Jun 2019 12:42:39 +0200
> Peter Zijlstra <peterz@...radead.org> wrote:
> 
> > No, the other way around, Sphinx can recognize local files and treat
> > them special. That way we keep the text readable.
> > 
> > Same with that :c:func:'foo' crap, that needs to die, and Sphinx needs
> > to be taught about foo().  
> 
> I did a patch to make that latter part happen, but haven't been able to
> find the time to address the comments and get it out there.  It definitely
> cleaned up the source files a lot and is worth doing.  Will try to get
> back to it soon.

See my comment. Yeah, the :c:func:'foo' (the version you merged at the automarkup
branch) has currently a bug, when there's something like:

	func()
	======

or when func() is inside a table.

Solving the table case would be a lot better if the plugin could run the
existing table parser and only then handle the cross-reference replacements,
but I've no idea how flexible the Sphinx plugins can be.

> 
> The local file links should be easy to do; we shouldn't need to add any
> markup for those.

Yeah, those are easy - except if someone adds a Documentation/* link 
inside a table or inside a topic header.

Running a modified version of your tool shows just two new warnings:

	Documentation/translations/ja_JP/howto.rst:176: WARNING: undefined label: :doc: (if the link has no caption the label must precede a section header)         
	Documentation/translations/zh_CN/process/submitting-drivers.rst:25: WARNING: unknown document: ../../../Documentation/translations/zh_CN/process/submitting-patches

The first one is because of this:

	:Ref:`Documentation/process/kernel-docs.rst <kernel_docs>`

(my parser didn't consider upper-case tags - a simple fix at a regex should
fix this)

The second one is because the URL is wrong. It is pointing to:

	Documentation/Documentation/translations/zh_CN/process/submitting-patches

at Chinese translation.

So, at least the way our documentation is, the plugin seems to be working
as expected.

As a reference, I'm enclosing the diff against your patch:

    commit 6231d7456e87bd3e11f892709945887bd55a8a20 (docs/automarkup)
    Author: Jonathan Corbet <corbet@....net>
    Date:   Thu Apr 25 07:55:07 2019 -0600

        Docs: An initial automarkup extension for sphinx
    
        Rather than fill our text files with :c:func:`function()` syntax, just do
        the markup via a hook into the sphinx build process.  As is always the
        case, the real problem is detecting the situations where this markup should
        *not* be done.
    
        Signed-off-by: Jonathan Corbet <corbet@....net>

Thanks,
Mauro

-

diff --git a/Documentation/sphinx/automarkup.py b/Documentation/sphinx/automarkup.py
index 39c8f4d5af82..60dad596c790 100644
--- a/Documentation/sphinx/automarkup.py
+++ b/Documentation/sphinx/automarkup.py
@@ -9,6 +9,7 @@
 from __future__ import print_function
 import re
 import sphinx
+#import sys		# Just for debug
 
 #
 # Regex nastiness.  Of course.
@@ -31,10 +32,26 @@ RE_literal = re.compile(r'^(\s*)(.*::\s*|\.\.\s+code-block::.*)$')
 #
 RE_whitesp = re.compile(r'^(\s*)')
 
+#
+# Get a documentation reference
+#
+RE_doc_links = re.compile(r'\bDocumentation/([\w\d\-\_\/]+)\.rst\b')
+
+#
+# Doc link false-positives
+#
+RE_false_doc_links = re.compile(r':ref:`\s*Documentation/[\w\d\-\_\/]+\.rst')
+
 def MangleFile(app, docname, text):
     ret = [ ]
     previous = ''
     literal = False
+
+    rel_dir = ''
+
+    for depth in range(0, docname.count('/')):
+        rel_dir += "../"
+
     for line in text[0].split('\n'):
         #
         # See if we might be ending a literal block, as denoted by
@@ -63,7 +80,18 @@ def MangleFile(app, docname, text):
         # Normal line - perform substitutions.
         #
         else:
-            ret.append(RE_function.sub(r'\1:c:func:`\2`\3', line))
+#            new_line = RE_function.sub(r'\1:c:func:`\2`\3', line)
+            new_line = line
+
+            if not RE_false_doc_links.search(new_line):
+                new_line = RE_doc_links.sub(r':doc:`' + rel_dir + r'\1`', new_line)
+
+ #           # Just for debug - should be removed on production
+ #           if new_line != line:
+ #               print ("===>" + new_line, file=sys.stderr)
+
+            ret.append(new_line)
+
         #
         # Might we be starting a literal block?  If so make note of
         # the fact.