linux-kernel - Re: [PATCH v4 00/52] Fix html build with Sphinx 3.1 and above

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20201002085222.41543304@coco.lan>
Date:   Fri, 2 Oct 2020 08:52:22 +0200
From:   Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To:     Jonathan Corbet <corbet@....net>
Cc:     Linux Doc Mailing List <linux-doc@...r.kernel.org>,
        linux-kernel@...r.kernel.org,
        "David S. Miller" <davem@...emloft.net>,
        "Rafael J. Wysocki" <rjw@...ysocki.net>,
        Alexander Aring <alex.aring@...il.com>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Anton Vorontsov <anton@...msg.org>,
        Colin Cross <ccross@...roid.com>,
        Daniel Lezcano <daniel.lezcano@...aro.org>,
        Daniel Thompson <daniel.thompson@...aro.org>,
        Daniel Vetter <daniel@...ll.ch>,
        David Airlie <airlied@...ux.ie>,
        Jakub Kicinski <kuba@...nel.org>,
        Jaroslav Kysela <perex@...ex.cz>,
        Jason Wessel <jason.wessel@...driver.com>,
        Kees Cook <keescook@...omium.org>,
        Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>,
        Mauro Carvalho Chehab <mchehab@...nel.org>,
        Maxime Ripard <mripard@...nel.org>,
        Mike Rapoport <rppt@...nel.org>,
        Moritz Fischer <mdf@...nel.org>, Sean Young <sean@...s.org>,
        Stefan Schmidt <stefan@...enfreihafen.org>,
        Takashi Iwai <tiwai@...e.com>, Tejun Heo <tj@...nel.org>,
        Thomas Zimmermann <tzimmermann@...e.de>,
        Tony Luck <tony.luck@...el.com>,
        Viresh Kumar <viresh.kumar@...aro.org>
Subject: Re: [PATCH v4 00/52] Fix html build with Sphinx  3.1 and above

Em Thu, 1 Oct 2020 15:48:54 -0600
Jonathan Corbet <corbet@....net> escreveu:

> On Wed, 30 Sep 2020 15:24:23 +0200
> Mauro Carvalho Chehab <mchehab+huawei@...nel.org> wrote:
> 
> > This series add proper support for Sphinx 3.1 and above for building the html docs.
> > 
> > This series comes after the one I wrote fixing the warnings. 
> > 
> > With this series applied on the top of next-20200922, there are just 12
> > warnings:
> > 	- 2 of them happens also on Sphinx 2.4.4
> > 	 (both seem easy to fix: I'll send later fixes for those);
> > 	- 10 happens only on Sphinx 3.2.1.
> > 
> > The new warnings are all due to duplicated C domain cross-reference symbols. 
> > 
> > Basically, the C domain on Sphinx doesn't allow to have an struct and
> > a function with the same name. I opened an issue on Sphinx.:
> > 
> >     https://github.com/sphinx-doc/sphinx/issues/8241
> > 
> > Hopefully, some newer version may have it fixed.
> > 
> > There is still one thing that requires a fix: the automarkup.py. The
> > way cross-references are stored with Sphinx 3 are different.
> > I didn't try yet to address it, but I suspect that it shouldn't be
> > hard to address it.  
> 
> Modulo my comment on part 4, I think this is what we want.  It's kind of
> unfortunate that it's all necessary, but that's the way things are these
> days, I guess.

Yes, it is unfortunate. Yet, my feeling is that this approach provides
us a good way to transition to the new C domain parser, which has some
advantages over the older code.

> This part is a bit intimidating, though:
> 
> >  288 files changed, 1709 insertions(+), 2183 deletions(-)  

Yes. Yet, the media userspace API is responsible for most of it:

 224 files changed, 1076 insertions(+), 1716 deletions(-)

As most of the uAPI documentation doesn't rely on kernel-doc
(mostly due to historic reasons).

To be fair, most of the other changes outside the building system
are due to already-existing issues at the documentation. There are
several kernel-doc tags creating duplicated C references.

While Sphinx < 3 C domain parser were able to detect them, such
warnings are disabled by default.

> Should we maybe position this as an end-of-merge-window blast, once other
> stuff has hopefully mostly settled?  I can certainly warn Linus that it's
> coming when I send the main docs pull.

Yeah, adding this series, together with the remaining patches fixing
all warnings with Sphinx < 3 at the end of the merge window seems
the right thing to do, IMO.

> I wonder how soon we could pull our minimum version forward to 3.1 and
> drop a bunch of stuff?  I fear it may not be for a while, alas...

I guess it may take some time. In any case, for the next minimal
version, I would set for the upcoming one that will fix those
issue reported here:

	https://github.com/sphinx-doc/sphinx/issues/7819
	https://github.com/sphinx-doc/sphinx/issues/8241

One such example is this warning:

	.../Documentation/driver-api/miscellaneous:48: ../drivers/pwm/core.c:669: WARNING: Duplicate C declaration, also defined in 'driver-api/miscellaneous'.
	Declaration is 'int pwm_capture (struct pwm_device *pwm, struct pwm_capture *result, unsigned long timeout)'.

	(See: we have a function pwm_capture and a struct with the
	 same name. Both have kernel-doc tags at drivers/pwm/core.c)

Without fixing it, there's no way to produce a zero-warning
docs building[1] with Sphinx versions above 2.4.x.

[1] Well, it could be possible to rename such kAPI symbols to
    make Sphinx happy, but IMO, the C domain there should be able
    to fully support the C language.

> Thanks for doing all this work,

Anytime.

Thanks,
Mauro