linux-kernel - Re: [RFC] doc: fix code snippet build warnings

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 16 Jan 2018 13:36:07 +0100
From:   Markus Heiser <markus.heiser@...marit.de>
To:     Jani Nikula <jani.nikula@...ux.intel.com>
Cc:     Jonathan Corbet <corbet@....net>, "Tobin C. Harding" <me@...in.cc>,
        "Paul E. McKenney" <paulmck@...ux.vnet.ibm.com>,
        Josh Triplett <josh@...htriplett.org>,
        Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [RFC] doc: fix code snippet build warnings


> Am 16.01.2018 um 11:22 schrieb Jani Nikula <jani.nikula@...ux.intel.com>:
> 
> On Wed, 10 Jan 2018, Jonathan Corbet <corbet@....net> wrote:
>> On Wed, 10 Jan 2018 15:04:53 +1100
>> "Tobin C. Harding" <me@...in.cc> wrote:
>> 
>>> Posting as RFC in the hope that someone knows how to massage sphinx
>>> correctly to fix this patch.
>>> 
>>> Currently function kernel-doc contains a multi-line code snippet. This
>>> is causing sphinx to emit 5 build warnings
>>> 
>>> 	WARNING: Unexpected indentation.
>>> 	WARNING: Unexpected indentation.
>>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>>> 	WARNING: Block quote ends without a blank line; unexpected unindent.
>>> 	WARNING: Inline literal start-string without end-string.
>>> 
>>> And the snippet is not rendering correctly in HTML.
>>> 
>>> We can stop shpinx complaining by using '::' instead of the currently
>>> used '``' however this still does not render correctly in HTML. The
>>> rendering is [arguably] better but still incorrect. Sphinx renders two
>>> function calls thus:
>>> 
>>> 	:c:func:`rcu_read_lock()`;
>>> 
>>> The rest of the snippet does however have correct spacing.
>> 
>> The behavior when `` was used is not surprising, that really just does a
>> font change.  Once you went with a literal block (with "::") though, the
>> situation changes a bit.  That really should work.
>> 
>> I looked a bit.  This isn't a sphinx (or "shpinx" :) problem, the bug is
>> in kernel-doc.  Once we go into the literal mode, it shouldn't be
>> screwing around with the text anymore.  Of course, kernel-doc doesn't
>> understand enough RST to know that.  I'm a little nervous about trying to
>> teach it more, but maybe we have to do that; we should certainly be able
>> to put code snippets into the docs and have them come through unmolested.
> 
> I think eventually the Right Thing would be to move most of kernel-doc's
> mucking of the comments (i.e. highlights) into kernel-doc the Sphinx
> extension. I've toyed with the idea, but it's not as trivial as I would
> like it to be.
> 
> Basically it involves flagging all the kernel-doc nodes during the read
> phase, and registering handlers to post process the doctrees. At that
> stage, it's no longer simple regexp replaces, it's replacing doctree
> nodes with ones that have reference nodes within them. It's more
> complicated,

Hi Jani, my thoughts about:  

1.) The reST syntax [1] (the parser part) is not *extendable*, we
can only add new roles [2] or directives.

2.) the kernel-doc syntax is weak and ambiguous. This
remains mainly in tagging only with a start-tag or only with a end-tag 
e.g:

* sectioning:   "Return:" --> end-tag just ":"
* fields:       "@arg1:"  --> better
* highlight/ref: start tag [@%$&] / no end tag

even if I don't know how (1.), if the highlight-syntax becomes a part
of the reST syntax we have to quote [@%$&] elsewhere. This will break
with valid reST documents which is unacceptable.

Thats why I think we need to pre-parse highlighting in the kernel-doc
Perl script, even if it is a bit hackish (how should it be otherwise,
if the syntax is already hackish). I haven't had the time to implement
such a highlight parser right now, but this would be the first shot of
mine. 

An alternative might be to use roles [2] but this means we have to
break with the the kernel-doc (highlight) syntax which is IMO also
unacceptable ... tricky situation :o

[1] http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html
[2] http://docutils.sourceforge.net/docs/howto/rst-roles.html


> but at that stage we can ignore stuff that should stay
> verbatim. I think it's also possible to check that the reference targets
> actually exist. In short, at that phase we have all the knowledge about
> the rst structure, parsed by Sphinx, and we don't have to duplicate that
> knowledge in kernel-doc the perl script.
> 
> Note that this has nothing to do with swithing to Python based
> kernel-doc suggested by Markus previously.

;)

-- Markus --