linux-kernel - Re: [PATCH 00/34] docs: avoid using ReST :doc:`foo` tag

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20210607093422.0a369909@coco.lan>
Date:   Mon, 7 Jun 2021 09:34:22 +0200
From:   Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To:     "Nícolas F. R. A. Prado" <n@...aprado.net>
Cc:     Jonathan Corbet <corbet@....net>,
        Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        linux-kernel@...r.kernel.org, bpf@...r.kernel.org,
        coresight@...ts.linaro.org, devicetree@...r.kernel.org,
        kunit-dev@...glegroups.com, kvm@...r.kernel.org,
        linux-acpi@...r.kernel.org, linux-arm-kernel@...ts.infradead.org,
        linux-gpio@...r.kernel.org, linux-hwmon@...r.kernel.org,
        linux-i2c@...r.kernel.org, linux-kselftest@...r.kernel.org,
        linux-media@...r.kernel.org, linux-pci@...r.kernel.org,
        linux-pm@...r.kernel.org, linux-security-module@...r.kernel.org,
        netdev@...r.kernel.org
Subject: Re: [PATCH 00/34] docs: avoid using ReST :doc:`foo` tag

Em Sun, 6 Jun 2021 19:52:25 -0300
Nícolas F. R. A. Prado <n@...aprado.net> escreveu:

> On Sat, Jun 05, 2021 at 09:08:36PM +0200, Mauro Carvalho Chehab wrote:
> > Em Sat, 5 Jun 2021 12:11:09 -0300
> > Nícolas F. R. A. Prado <n@...aprado.net> escreveu:
> >   
> > > Hi Mauro,
> > > 
> > > On Sat, Jun 05, 2021 at 03:17:59PM +0200, Mauro Carvalho Chehab wrote:  
> > > > As discussed at:
> > > > 	https://lore.kernel.org/linux-doc/871r9k6rmy.fsf@meer.lwn.net/
> > > > 
> > > > It is better to avoid using :doc:`foo` to refer to Documentation/foo.rst, as the
> > > > automarkup.py extension should handle it automatically, on most cases.
> > > > 
> > > > There are a couple of exceptions to this rule:
> > > > 
> > > > 1. when :doc:  tag is used to point to a kernel-doc DOC: markup;
> > > > 2. when it is used with a named tag, e. g. :doc:`some name <foo>`;
> > > > 
> > > > It should also be noticed that automarkup.py has currently an issue:
> > > > if one use a markup like:
> > > > 
> > > > 	Documentation/dev-tools/kunit/api/test.rst
> > > > 	  - documents all of the standard testing API excluding mocking
> > > > 	    or mocking related features.
> > > > 
> > > > or, even:
> > > > 
> > > > 	Documentation/dev-tools/kunit/api/test.rst
> > > > 	    documents all of the standard testing API excluding mocking
> > > > 	    or mocking related features.
> > > > 	
> > > > The automarkup.py will simply ignore it. Not sure why. This patch series
> > > > avoid the above patterns (which is present only on 4 files), but it would be
> > > > nice to have a followup patch fixing the issue at automarkup.py.    
> > > 
> > > What I think is happening here is that we're using rST's syntax for definition
> > > lists [1]. automarkup.py ignores literal nodes, and perhaps a definition is
> > > considered a literal by Sphinx. Adding a blank line after the Documentation/...
> > > or removing the additional indentation makes it work, like you did in your
> > > 2nd and 3rd patch, since then it's not a definition anymore, although then the
> > > visual output is different as well.  
> > 
> > A literal has a different output. I think that this is not the case, but I 
> > didn't check the python code from docutils/Sphinx.  
> 
> Okay, I went in deeper to understand the issue and indeed it wasn't what I
> thought. The reason definitions are ignored by automarkup.py is because the main
> loop iterates only over nodes that are of type paragraph:
> 
>     for para in doctree.traverse(nodes.paragraph):
>         for node in para.traverse(nodes.Text):
>             if not isinstance(node.parent, nodes.literal):
>                 node.parent.replace(node, markup_refs(name, app, node))
> 
> And inspecting the HTML output from your example, the definition name is inside
> a <dt> tag, and it doesn't have a <p> inside. So in summary, automarkup.py will
> only work on elements which are inside a <p> in the output.


Yeah, that's what I was suspecting, based on the comments.

Maybe something similar to the above could be done also for some
non-paragraph data. By looking at:

	https://docutils.sourceforge.io/docs/ref/doctree.html

It says that the body elements are:

	admonition, attention, block_quote, bullet_list, caution, citation, 
	comment, compound, container, danger, definition_list, doctest_block, 
	enumerated_list, error, field_list, figure, footnote, hint, image, 
	important, line_block, literal_block, note, option_list, paragraph, 
	pending, raw, rubric, substitution_definition, system_message, 
	table, target, tip, warning

So, perhaps a similar loop for definition_list would do the trick,
but maybe automarkup should also look at other types, like enum lists,
notes (and their variants, like error/warning) and footnotes.

No idea how this would affect the docs build time, though.

> Only applying the automarkup inside paragraphs seems like a good decision (which
> covers text in lists and tables as well), so unless there are other types of
> elements without paragraphs where automarkup should work, I think we should just
> avoid using definition lists pointing to documents like that.

Checking the code or doing some tests are needed for us to be sure about what
of the above types docutils don't consider a paragraph.

Thanks,
Mauro