Date:	Tue, 7 Jun 2016 09:54:21 +0200
From:	Daniel Vetter <daniel.vetter@...ll.ch>
To:	Markus Heiser <markus.heiser@...marit.de>
Cc:	Jonathan Corbet <corbet@....net>,
	Jani Nikula <jani.nikula@...el.com>,
	Grant Likely <grant.likely@...retlab.ca>,
	Mauro Carvalho Chehab <mchehab@....samsung.com>,
	Keith Packard <keithp@...thp.com>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>,
	linux-doc@...r.kernel.org, Hans Verkuil <hverkuil@...all.nl>
Subject: Re: [PATCH 0/7] add reST/sphinx-doc to linux documentation

On Mon, Jun 6, 2016 at 6:32 PM, Markus Heiser <markus.heiser@...marit.de> wrote:
> From: "Heiser, Markus" <markus.heiser@...marIT.de>
>
> Hi Jonathan, Jani, all -
>
> I merged the work from sphkerneldoc POC [1], into branch (on v4.7-rc2)
>
>     git://github.com/return42/linux.git linux-doc-reST
>
> The specification of the implementation is the (new) kernel-doc-HOWTO book which
> is a part of the patch series (also online avaliable at [2]).
>
> The kernel-doc-HOWTO book is a rewrite of the kernel-doc-nano-HOWTO.txt,
> taking reST extension into account. It serves several purposes:
>
> * user guide for kernel developers
> * specification of the kernel-doc parser
> * test case
>
> The kernel_doc.py parser is a python implementation of the kernel-doc-HOWTO, the
> kernel-doc perl script is not needed for reST builds.  The python variant of the
> kernel-doc parser supports the markup modes:
>
>  * kernel-doc (vintage)
>  * reST
>
> This series also includes the reST directives describe in the kernel-doc-HOWTO:
>
> * kernel-doc (uses/imports the parser from kernel_doc.py)
> * flat-table (table notation with meaningfull diffs)
>
> The Python/Sphinx extensions placed in the scripts/site-packages folder. With
> this folder a minimal python infrastructure is provided.
>
> The basic sphinx-build infrastructure consists of:
>
> * Documentation/Makefile.reST & Documentation/conf.py
>
>   Makefile and basic 'sphinx config' file to build the various reST
>   documents and output formats. Provides the basic sphinx-doc build
>   infrastructure including the *sphinx-subprojects* feature. With this
>   feature each book can be build and distributed stand-alone.
>
> * Documentation/books
>
>   In this folder, the books with reST markup are placed. To
>   provide *sphinx-subprojects*, each book has its one folder and
>   a (optional) Documentation/books/{book-name}.conf file
>   which *overwrites* the basic configuration from Documentation/conf.py
>
> Any comments are welcome, but please read [2] first.

I think getting the flat-table directive landed would be great. We
have a similarly annoying table in gpu.tmpl where the ascii-art tables
fall short and are annoying. We want to split it up, but that's not a
short-term solution suitable for conversion. Atm it's licensed under
gplv3, which is incompatible with the kernel, so need to fix that.

I'm still not sold on the vintage-kerneldoc idea. At least in my
experience (after first converting to asciidoc and now to sphinx) this
is a non-issue. Yes, there's the oddball misrendering, but that's no
worse than the oddball typo. And a really good way for
drive-by-contributors and newbies to send in a few useful patches.
Personally I'm totally happy to have that bit of ugliness, same way I
don't reject patches just because they trip over checkpatch.pl in some
minor detail. The downside otoh means we'll have to forever maintain 2
versions of kernel-doc, and I don't want that. Jani&me have already
ripped out some of the more bonghits features of kernel-doc (like
anything with a \w: is a heading and other nonsense like that). The
only thing we really need to keep as special is the overall kerneldoc
layout, and the special referencing using &, %, @ and friends.

Reimplementing kernel-doc in python is also something Jani&I
discussed. But I think it only makes sense if we go ahead and also
modernize the parser, i.e. use something like python-clang to parse C,
and build a modern parser-combinators thing for the kernel-doc itself.
The regex state machinery is a horror show, whether it's written in
cargo-culted perl or python.

I agree that we need to update the kernel-doc howto (and Jani's
working on that already too), but I think there's no need to split up
that finely. Maybe that's a bit against sphinx best practices, but
right now we have rather large documents (both plain text and
docbook). Splitting definitely makes sense in some cases (e.g.
gpu.tmpl is only this large because we couldn't do cross-template
referencing), but by-and-large I think the current splitting is mostly
reasonable. A better demonstration vehicle is imo converting all
existing .tmpl to .rst. And also making sure the new kernel-doc is
still able to parse all existing kernel-doc comments, even when
they're not pulled into .tmpl files. Jani has done that (using
hacked-up versions of the sphinx-lint script) for his tree, making
sure there's no unexpected changes anywhere.

I think next steps would be:
- rebase flat-table onto Jani's work and relicense under gplv2
- look into rewriting kernel-doc in python more as a long-term project
- start converting docs instead - I really want to start reaping
benefits of all this work as soon as possible.

Thanks, Daniel


>
> With regards
>
>   -- Markus --
>
> [1] https://return42.github.io/sphkerneldoc
> [2] https://return42.github.io/sphkerneldoc/books/kernel-doc-HOWTO
>
>
>
> Heiser, Markus (7):
>   python: add scripts/site-packages
>   sphinx-doc: add basic sphinx-build infrastructure
>   kernel-doc-HOWTO: add kernel-doc specification
>   linuxdoc: add python package linuxdoc
>   kernel-doc parser: inital python implementation
>   kernel-doc directive: initial implementation
>   flat-table directive: initial implementation
>
>  Documentation/.gitignore                           |    2 +
>  Documentation/Makefile.reST                        |  111 +
>  Documentation/books/.gitignore                     |    0
>  Documentation/books/kernel-doc-HOWTO.conf          |  200 ++
>  .../books/kernel-doc-HOWTO/all-in-a-tumble-src.rst |   12 +
>  .../books/kernel-doc-HOWTO/all-in-a-tumble.h       |  271 ++
>  .../books/kernel-doc-HOWTO/all-in-a-tumble.rst     |   11 +
>  Documentation/books/kernel-doc-HOWTO/csv_table.txt |    6 +
>  Documentation/books/kernel-doc-HOWTO/index.rst     |   46 +
>  .../kernel-doc-HOWTO/kernel-doc-components.rst     |   35 +
>  .../kernel-doc-HOWTO/kernel-doc-directive.rst      |  199 ++
>  .../books/kernel-doc-HOWTO/kernel-doc-examples.rst |   16 +
>  .../books/kernel-doc-HOWTO/kernel-doc-intro.rst    |   85 +
>  .../books/kernel-doc-HOWTO/kernel-doc-syntax.rst   |  171 ++
>  .../kernel-doc-HOWTO/reST-kernel-doc-mode.rst      |  120 +
>  Documentation/books/kernel-doc-HOWTO/refs.txt      |   15 +
>  .../books/kernel-doc-HOWTO/table-markup.rst        |  499 ++++
>  .../books/kernel-doc-HOWTO/test/parser_test.h      |  332 +++
>  .../kernel-doc-HOWTO/vintage-kernel-doc-mode.rst   |  100 +
>  Documentation/conf.py                              |  357 +++
>  Documentation/sphinx-static/theme_overrides.css    |   45 +
>  Documentation/sphinx-tex/Makefile                  |   88 +
>  scripts/site-python/.gitignore                     |    1 +
>  scripts/site-python/README                         |   14 +
>  scripts/site-python/linuxdoc/__init__.py           |    8 +
>  scripts/site-python/linuxdoc/kernel_doc.py         | 2764 ++++++++++++++++++++
>  scripts/site-python/linuxdoc/rstFlatTable.py       |  356 +++
>  scripts/site-python/linuxdoc/rstKernelDoc.py       |  369 +++
>  28 files changed, 6233 insertions(+)
>  create mode 100644 Documentation/.gitignore
>  create mode 100644 Documentation/Makefile.reST
>  create mode 100644 Documentation/books/.gitignore
>  create mode 100644 Documentation/books/kernel-doc-HOWTO.conf
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble-src.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.h
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/all-in-a-tumble.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/csv_table.txt
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/index.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-components.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-directive.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-examples.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-intro.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/kernel-doc-syntax.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/reST-kernel-doc-mode.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/refs.txt
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/table-markup.rst
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/test/parser_test.h
>  create mode 100644 Documentation/books/kernel-doc-HOWTO/vintage-kernel-doc-mode.rst
>  create mode 100644 Documentation/conf.py
>  create mode 100644 Documentation/sphinx-static/theme_overrides.css
>  create mode 100644 Documentation/sphinx-tex/Makefile
>  create mode 100644 scripts/site-python/.gitignore
>  create mode 100644 scripts/site-python/README
>  create mode 100644 scripts/site-python/linuxdoc/__init__.py
>  create mode 100755 scripts/site-python/linuxdoc/kernel_doc.py
>  create mode 100755 scripts/site-python/linuxdoc/rstFlatTable.py
>  create mode 100755 scripts/site-python/linuxdoc/rstKernelDoc.py
>
> --
> v4.7-rc-2
>



-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch

Powered by blists - more mailing lists