linux-kernel - Re: [PATCH v2] Documentation: Document each netlink family

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <ZVu5rq1SdloY41nH@gmail.com>
Date:   Mon, 20 Nov 2023 11:55:26 -0800
From:   Breno Leitao <leitao@...ian.org>
To:     Jakub Kicinski <kuba@...nel.org>
Cc:     leit@...a.com, Jonathan Corbet <corbet@....net>,
        netdev@...r.kernel.org, donald.hunter@...il.com,
        linux-doc@...r.kernel.org, pabeni@...hat.com, edumazet@...gle.com,
        linux-kernel@...r.kernel.org
Subject: Re: [PATCH v2] Documentation: Document each netlink family

On Fri, Nov 17, 2023 at 04:39:39PM -0800, Jakub Kicinski wrote:
> On Fri, 17 Nov 2023 15:17:02 -0700 Jonathan Corbet wrote:
> > In principle I like this approach better.  There is one problem, though:
> > 
> > - In current kernels, on my machine, "make htmldocs" when nothing has
> >   changed takes about 6s to complete.
> > 
> > - With this patch applied, it takes a little over 5 *minutes*.
> > 
> > Without having delved into it too far, I am guessing that the
> > unconditional recreation of the netlink RST files is causing the rebuild
> > of much of the documentation.  Even so, I don't quite get it.
> > 
> > That, clearly, would need to be fixed before this can go in.
> 
> FWIW on the C code-gen side we avoid touching the files if nothing
> changed both at the Makefile level:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/tree/tools/net/ynl/generated/Makefile#n28

I was able to do something similar and it works quite well. Only
regenerate what is not up-to-date. See below what I am doing. (I needed
to change the python to adapt)

> And the tool itself actually generates to a tempfile and compares
> if the output changed:
> 
> https://git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/?id=2b7ac0c87d985c92e519995853c52b9649ea4b07

I am not planning to do it, since I would like to trust Make. Let me
know if you think this is important and I can do it also.

--
diff --git a/Documentation/Makefile b/Documentation/Makefile
index 2f35793acd2a..dad6e2ecf082 100644
--- a/Documentation/Makefile
+++ b/Documentation/Makefile
@@ -97,7 +97,22 @@ quiet_cmd_sphinx = SPHINX  $@ --> file://$(abspath $(BUILDDIR)/$3/$4)
                cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_stati
c/; \
        fi

-htmldocs:
+YNL_TOOL:=$(srctree)/tools/net/ynl/ynl-gen-rst.py
+YNL_YAML_DIR:=$(srctree)/Documentation/netlink/specs
+YNL_RST_DIR:=$(srctree)/Documentation/networking/netlink_spec
+YNL_INDEX:=$(srctree)/Documentation/networking/netlink_spec/index.rst
+
+YNL_YAML_FILES := $(wildcard $(YNL_YAML_DIR)/*.yaml)
+YNL_RST_FILES_TMP := $(patsubst %.yaml,%.rst,$(wildcard $(YNL_YAML_DIR)/*.yaml))
+YNL_RST_FILES := $(patsubst $(YNL_YAML_DIR)%,$(YNL_RST_DIR)%, $(YNL_RST_FILES_TMP))
+
+$(YNL_INDEX): $(YNL_RST_FILES)
+       $(YNL_TOOL) -x # Generate the index
+
+%.rst: $(YNL_YAMLS_FILES)
+       $(YNL_TOOL) -i $(patsubst %.rst,%.yaml, $(@F)) # generate individual rst files
+
+htmldocs: $(YNL_INDEX)
        @$(srctree)/scripts/sphinx-pre-install --version-check
        @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var)))