linux-kernel - Re: Kernel docs: muddying the waters a bit

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <13D877B1-B9A2-412A-BA43-C6A5B881A536@darmarit.de>
Date:	Wed, 4 May 2016 14:40:29 +0200
From:	Markus Heiser <markus.heiser@...marit.de>
To:	Jani Nikula <jani.nikula@...el.com>
Cc:	Daniel Vetter <daniel.vetter@...ll.ch>,
	Jonathan Corbet <corbet@....net>,
	Grant Likely <grant.likely@...retlab.ca>,
	Mauro Carvalho Chehab <mchehab@....samsung.com>,
	Dan Allen <dan@...ndevise.io>,
	Russel Winder <russel@...der.org.uk>,
	Keith Packard <keithp@...thp.com>,
	LKML <linux-kernel@...r.kernel.org>, linux-doc@...r.kernel.org,
	Hans Verkuil <hverkuil@...all.nl>,
	"linux-media\@vger.kernel.org linux-media" 
	<linux-media@...r.kernel.org>,
	Graham Whaley <graham.whaley@...ux.intel.com>
Subject: Re: Kernel docs: muddying the waters a bit

Hi Jani,

Am 04.05.2016 um 11:58 schrieb Jani Nikula <jani.nikula@...el.com>:

> On Wed, 04 May 2016, Markus Heiser <markus.heiser@...marit.de> wrote:
>> but I think this will not by very helpful, as long as you miss 
>> a similar ".tmpl" workflow for reST documents.
>> 
>> I'am working on a reST directive (named: "kernel-doc") to provide a
>> similar ".tmpl" workflow within plain reST. The first step towards
>> is done with (my) modified kernel-doc script ...
>> 
>> * https://github.com/return42/sphkerneldoc/blob/master/scripts/kernel-doc#L1736
>> 
>> which produce reST from source code comments. E.g. this content is 
>> generated with it.
> 
> What do you mean by ".tmpl workflow"?

Sorry for bad naming, I addressed the DOCPROC build process which builds
the .xml files from .tmpl files ...

<SNIP> ----
###
# The build process is as follows (targets):
#              (xmldocs) [by docproc]
# file.tmpl --> file.xml +--> file.ps   (psdocs)   [by db2ps or xmlto]
#                        +--> file.pdf  (pdfdocs)  [by db2pdf or xmlto]
#                        +--> DIR=file  (htmldocs) [by xmlto]
#                        +--> man/      (mandocs)  [by xmlto]
....
###
# DOCPROC is used for two purposes:
# 1) To generate a dependency list for a .tmpl file
# 2) To preprocess a .tmpl file and call kernel-doc with
#     appropriate parameters.
# The following rules are used to generate the .xml documentation
# required to generate the final targets. (ps, pdf, html).
<SNAP> -----


The DOCPROC markup directives are described in the kernel-doc-nano-HOWTO:

https://github.com/torvalds/linux/blob/master/Documentation/kernel-doc-nano-HOWTO.txt#L332

My aim is to implement a reST-directive which is similar. E.g: 
 
<XML-SNIP> -----
    <sect2>
      <title>Device Instance and Driver Handling</title>
!Pdrivers/gpu/drm/drm_drv.c driver instance overview
!Edrivers/gpu/drm/drm_drv.c
    </sect2>
    <sect2>
<XML-SNAP> -----

In reST the directive might look like:

<reST-SNIP> -----
Device Instance and Driver Handling
===================================

.. kernel-doc::  drivers/gpu/drm/drm_drv.c
   :doc:      driver instance overview
   :exported:

<reST-SNAP> -----


> I'd be *very* hesitant about adding the kind of things you do in
> reformat_block_rst to kernel-doc. IMO the extraction from kernel-doc
> comments must be as simple as possible with basically pass-through of
> the comment blocks to sphinx.

Right, but you forgot, that the current markup in the source code comments
is based on the  kernel-doc-nano-HOWTO and I guess no one will migrate all
these source code comments to reST markup ;-)

So there is a need to differentiate between *vintage* kernel-doc markup 
and reST markup.

My suggestion is to add a tag to the comments, here a short example
what this might look like.

<vintage-comment-SNIP> --------------
/**
 * drm_minor_release - Release DRM minor
 * @minor: Pointer to DRM minor object
 *
 * Release a minor that was previously acquired via drm_minor_acquire().
 */
<vintage-comment-SNAP> --------------

in short: the vintage style does not need any change and 
comments with reST markup has a tag ":reST:" in the first 
line(s) ...

<reST-comment-SNIP> --------------
/**
 * :reST:
 *
 * .. c:function:: void drm_minor_release(struct drm_minor *minor)
 *
 *    Release DRM minor
 *
 *    :param struct drm_minor \*minor: Pointer to DRM minor object
 *
 */
<reST-comment-SNAP> --------------

Comments with the ":reST:" tag could be exported and pass-through
to sphinx.

> Specifically, do not attempt to detect and
> parse elements like lists in kernel-doc.

If your markup in the comments is plain reST, no need to parse, but there
are markups in the (vintage) comments which made use of individual 
text-markups, e.g. the markup of lists or ASCII-art. 

This individual text-markups has not been converted/rendered in the docbook
output, but I wanted to convert this individuals to reST, to render them in
Sphinx.

E.g. compare the member & description section of struct drm-display-mode ...

DocBook: 

  * https://www.kernel.org/doc/htmldocs/gpu/API-struct-drm-display-mode.html

kernel-doc reST with the additional reformat_block_rst function:

  * http://return42.github.io/sphkerneldoc/linux_src_doc/include/drm/drm_modes_h.html#struct-drm-display-mode


-- Markus--

> -- 
> Jani Nikula, Intel Open Source Technology Center
> --
> To unsubscribe from this list: send the line "unsubscribe linux-media" in
> the body of a message to majordomo@...r.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html