| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <20150630170138.GC30960@phenom.ffwll.local>
Date: Tue, 30 Jun 2015 19:01:38 +0200
From: Daniel Vetter <daniel@...ll.ch>
To: Danilo Cesar Lemes de Paula <danilo.cesar@...labora.co.uk>
Cc: linux-doc@...r.kernel.org, Randy Dunlap <rdunlap@...radead.org>,
Daniel Vetter <daniel.vetter@...ll.ch>,
Laurent Pinchart <laurent.pinchart@...asonboard.com>,
Jonathan Corbet <corbet@....net>,
Herbert Xu <herbert@...dor.apana.org.au>,
Stephan Mueller <smueller@...onox.de>,
Michal Marek <mmarek@...e.cz>, linux-kernel@...r.kernel.org,
intel-gfx <intel-gfx@...ts.freedesktop.org>,
dri-devel <dri-devel@...ts.freedesktop.org>
Subject: Re: [PATCH v2] scripts/kernel-doc: Adding cross-reference links to
html documentation.
On Fri, Jun 26, 2015 at 12:08:57PM -0300, Danilo Cesar Lemes de Paula wrote:
> Functions, Structs and Parameters definitions on kernel documentation
> are pure cosmetic, it only highlights the element.
>
> To ease the navigation in the documentation we should use <links> inside
> those tags so readers can easily jump between methods directly.
>
> This was discussed in 2014[1] and is implemented by getting a list
> of <refentries> from the DocBook XML to generate a database. Then it looks
> for <function>,<structnames> and <paramdef> tags that matches the ones in
> the database. As it only links existent references, no broken links are
> added.
>
> [1] - lists.freedesktop.org/archives/dri-devel/2014-August/065404.html
>
> Signed-off-by: Danilo Cesar Lemes de Paula <danilo.cesar@...labora.co.uk>
> Cc: Randy Dunlap <rdunlap@...radead.org>
> Cc: Daniel Vetter <daniel.vetter@...ll.ch>
> Cc: Laurent Pinchart <laurent.pinchart@...asonboard.com>
> Cc: Jonathan Corbet <corbet@....net>
> Cc: Herbert Xu <herbert@...dor.apana.org.au>
> Cc: Stephan Mueller <smueller@...onox.de>
> Cc: Michal Marek <mmarek@...e.cz>
> Cc: linux-kernel@...r.kernel.org
> Cc: linux-doc@...r.kernel.org
> Cc: intel-gfx <intel-gfx@...ts.freedesktop.org>
> Cc: dri-devel <dri-devel@...ts.freedesktop.org>
I'm using this now to generate the drm docbook and I think it's a great
improvement for them. No r-b since I can't really perl.
Acked-by: Daniel Vetter <daniel.vetter@...ll.ch>
> ---
> To understand a bit more of what this patch is trying to acomplish you can find
> two examples of the old and new htmldocs outputs:
> OLD: https://people.collabora.com/~danilo/intel/Documentation.old/DocBook/drm/API-drm-crtc-vblank-on.html
> NEW: https://people.collabora.com/~danilo/intel/Documentation.new/DocBook/drm/API-drm-crtc-vblank-on.html
>
> Documentation/DocBook/Makefile | 34 +++++---
> scripts/kernel-doc-xml-ref | 181 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 202 insertions(+), 13 deletions(-)
> create mode 100755 scripts/kernel-doc-xml-ref
>
> diff --git a/Documentation/DocBook/Makefile b/Documentation/DocBook/Makefile
> index b6a6a2e..8aea45a 100644
> --- a/Documentation/DocBook/Makefile
> +++ b/Documentation/DocBook/Makefile
> @@ -64,8 +64,9 @@ installmandocs: mandocs
>
> ###
> #External programs used
> -KERNELDOC = $(srctree)/scripts/kernel-doc
> -DOCPROC = $(objtree)/scripts/docproc
> +KERNELDOCXMLREF = $(srctree)/scripts/kernel-doc-xml-ref
> +KERNELDOC = $(srctree)/scripts/kernel-doc
> +DOCPROC = $(objtree)/scripts/docproc
>
> XMLTOFLAGS = -m $(srctree)/$(src)/stylesheet.xsl
> XMLTOFLAGS += --skip-validation
> @@ -89,7 +90,7 @@ define rule_docproc
> ) > $(dir $@).$(notdir $@).cmd
> endef
>
> -%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) FORCE
> +%.xml: %.tmpl $(KERNELDOC) $(DOCPROC) $(KERNELDOCXMLREF) FORCE
> $(call if_changed_rule,docproc)
>
> # Tell kbuild to always build the programs
> @@ -139,8 +140,12 @@ quiet_cmd_db2html = HTML $@
> cmd_db2html = xmlto html $(XMLTOFLAGS) -o $(patsubst %.html,%,$@) $< && \
> echo '<a HREF="$(patsubst %.html,%,$(notdir $@))/index.html"> \
> $(patsubst %.html,%,$(notdir $@))</a><p>' > $@
> +%.aux.xml: %.xml
> + @rm -rf $@
> + (cat $< | egrep "^<refentry id" | egrep -o "\".*\"" | cut -f 2 -d \" > $<.db)
> + $(KERNELDOCXMLREF) -db $<.db $< > $@
>
> -%.html: %.xml
> +%.html: %.aux.xml
> @(which xmlto > /dev/null 2>&1) || \
> (echo "*** You need to install xmlto ***"; \
> exit 1)
> @@ -209,15 +214,18 @@ dochelp:
> ###
> # Temporary files left by various tools
> clean-files := $(DOCBOOKS) \
> - $(patsubst %.xml, %.dvi, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.aux, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.tex, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.log, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.out, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.ps, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.pdf, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.html, $(DOCBOOKS)) \
> - $(patsubst %.xml, %.9, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.dvi, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.aux, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.tex, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.log, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.out, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.ps, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.pdf, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.html, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.9, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.aux.xml, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.xml.db, $(DOCBOOKS)) \
> + $(patsubst %.xml, %.xml, $(DOCBOOKS)) \
> $(index)
>
> clean-dirs := $(patsubst %.xml,%,$(DOCBOOKS)) man
> diff --git a/scripts/kernel-doc-xml-ref b/scripts/kernel-doc-xml-ref
> new file mode 100755
> index 0000000..c61cdad
> --- /dev/null
> +++ b/scripts/kernel-doc-xml-ref
> @@ -0,0 +1,181 @@
> +#!/usr/bin/perl -w
> +
> +use strict;
> +
> +## Copyright (C) 2015 Intel Corporation ##
> +# ##
> +## This software falls under the GNU General Public License. ##
> +## Please read the COPYING file for more information ##
> +#
> +#
> +# This software reads a XML file and a list of valid interal
> +# references to replace Docbook tags with links.
> +#
> +# usage:
> +# kernel-doc-xml-ref -db filename
> +# xml filename > outputfile
> +
> +# read arguments
> +if ($#ARGV != 2) {
> + usage();
> +}
> +
> +#Holds the database filename
> +my $databasefile;
> +my @database;
> +
> +#holds the inputfile
> +my $inputfile;
> +my $errors = 0;
> +
> +my %highlights = (
> + "<function>(.*?)</function>",
> + "\"<function>\" . convert_function(\$1, \$line) . \"</function>\"",
> + "<structname>(.*?)</structname>",
> + "\"<structname>\" . convert_struct(\$1) . \"</structname>\"",
> + "<funcdef>(.*?)<function>(.*?)</function></funcdef>",
> + "\"<funcdef>\" . convert_param(\$1) . \"<function>\$2</function></funcdef>\"",
> + "<paramdef>(.*?)<parameter>(.*?)</parameter></paramdef>",
> + "\"<paramdef>\" . convert_param(\$1) . \"<parameter>\$2</parameter></paramdef>\"");
> +
> +while($ARGV[0] =~ m/^-(.*)/) {
> + my $cmd = shift @ARGV;
> + if ($cmd eq "-db") {
> + $databasefile = shift @ARGV
> + }
> +}
> +$inputfile = shift @ARGV;
> +
> +sub open_database {
> + open my $handle, '<', $databasefile;
> + chomp(my @lines = <$handle>);
> + close $handle;
> +
> + @database = @lines;
> +}
> +
> +sub process_file {
> + open_database();
> +
> + my $dohighlight;
> + foreach my $pattern (keys %highlights) {
> + $dohighlight .= "\$line =~ s:$pattern:$highlights{$pattern}:eg;\n";
> + }
> +
> + open(FILE, $inputfile) or die("Could not open $inputfile");
> + foreach my $line (<FILE>) {
> + eval $dohighlight;
> + print $line;
> + }
> +}
> +
> +sub trim($_)
> +{
> + my $str = $_[0];
> + $str =~ s/^\s+|\s+$//g;
> + return $str
> +}
> +
> +sub has_key_defined($_)
> +{
> + if ( grep( /^$_[0]$/, @database)) {
> + return 1;
> + }
> + return 0;
> +}
> +
> +sub convert_function($_)
> +{
> + my $arg = $_[0];
> + my $key = $_[0];
> +
> + my $line = $_[1];
> +
> + $key = trim($key);
> +
> + $key =~ s/[^A-Za-z0-9]/-/g;
> + $key = "API-" . $key;
> +
> + # We shouldn't add links to <funcdef> prototype
> + if (!has_key_defined($key) || $line =~ m/\s+<funcdef/i) {
> + return $arg;
> + }
> +
> + my $head = $arg;
> + my $tail = "";
> + if ($arg =~ /(.*?)( ?)$/) {
> + $head = $1;
> + $tail = $2;
> + }
> + return "<link linkend=\"$key\">$head</link>$tail";
> +}
> +sub convert_struct($_)
> +{
> + my $arg = $_[0];
> + my $key = $_[0];
> + $key =~ s/(struct )?(\w)/$2/g;
> + $key =~ s/[^A-Za-z0-9]/-/g;
> + $key = "API-struct-" . $key;
> +
> + if (!has_key_defined($key)) {
> + return $arg;
> + }
> +
> + my ($head, $tail) = split_pointer($arg);
> + return "<link linkend=\"$key\">$head</link>$tail";
> +}
> +
> +sub split_pointer($_)
> +{
> + my $arg = $_[0];
> + if ($arg =~ /(.*?)( ?\* ?)/) {
> + return ($1, $2);
> + }
> + return ($arg, "");
> +}
> +
> +sub convert_param($_)
> +{
> + my $type = $_[0];
> + my $keyname = convert_key_name($type);
> +
> + if (!has_key_defined($keyname)) {
> + return $type;
> + }
> +
> + my ($head, $tail) = split_pointer($type);
> + return "<link linkend=\"$keyname\">$head</link>$tail";
> +
> +}
> +
> +sub convert_key_name($_)
> +{
> + #Pattern $2 is optional and might be uninitialized
> + no warnings 'uninitialized';
> +
> + my $str = $_[0];
> + $str =~ s/(const|static)? ?(struct)? ?([a-zA-Z0-9_]+) ?(\*|&)?/$2 $3/g ;
> +
> + # trim
> + $str =~ s/^\s+|\s+$//g;
> +
> + # spaces and _ to -
> + $str =~ s/[^A-Za-z0-9]/-/g;
> +
> + return "API-" . $str;
> +}
> +
> +sub usage {
> + print "Usage: $0 -db database filename ]\n";
> + print " xml source file(s) > outputfile\n";
> + exit 1;
> +}
> +
> +# starting point
> +process_file();
> +
> +if ($errors) {
> + print STDERR "$errors errors\n";
> +}
> +
> +exit($errors);
> --
> 2.1.4
>
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Powered by blists - more mailing lists