linux-kernel - Re: [PATCH v2] docs: automarkup.py: Allow automatic cross-reference inside C namespace

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <C76SLA2MZ7PJ.1I3UT6HA93TAK@ArchWay>
Date:   Wed, 18 Nov 2020 23:53:02 +0000
From:   Nícolas F. R. A. Prado 
        <nfraprado@...tonmail.com>
To:     Jonathan Corbet <corbet@....net>
Cc:     Mauro Carvalho Chehab <mchehab+huawei@...nel.org>,
        linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
        lkcamp@...ts.libreplanetbr.org, andrealmeid@...labora.com
Subject: Re: [PATCH v2] docs: automarkup.py: Allow automatic cross-reference inside C namespace

On Wed Nov 18, 2020 at 5:37 PM -03, Jonathan Corbet wrote:
>
> On Tue, 17 Nov 2020 02:12:01 +0000
> Nícolas F. R. A. Prado <nfraprado@...tonmail.com> wrote:
>
> > Sphinx 3.1 introduced namespaces for C cross-references. With this,
> > each C domain type/function declaration is put inside the namespace that
> > was active at the time of its declaration.
> >
> > Add support for automatic cross-referencing inside C namespaces by
> > checking whether the corresponding source file had a C namespace Sphinx
> > directive, and if so, try cross-referencing inside of it before going to
> > the global scope.
> >
> > This assumes there's only one namespace (if any) per rst file.
> >
> > Signed-off-by: Nícolas F. R. A. Prado <nfraprado@...tonmail.com>
> > ---
> >
> > To those following from v1:
> >
> > I ended up doing the simplest solution possible, which is to just directly read
> > the rst source corresponding to the doc page right before doing the automarkup.
> > It's not very efficient in the sense that the source is being read
> > twice (first by Sphinx, then by this), but it sidesteps the "data sharing
> > between processes" issue, so parallel_read_safe can be reenabled, and I didn't
> > notice any performance hit from this patch (as opposed to the big hit from v1).
> > Works with both Sphinx 2 and 3.
>
> The solution does lack elegance, but it is a solution, which is more than
> we had before :)

Exactly :P

> That said, rather than re-opening and re-reading the
> file, why not just connect to the source-read event, which will happily
> hand you the document source that it has already read?

Unfortunately that wouldn't work. What would happen is that Sphinx would spawn a
python process to handle the source-read event for file X, then later it would
spawn another python process to handle doctree-resolved for the same file X.
Being two different processes, data can't easily be shared between them, which
is why I originally disabled parallel_read_safe, to force everything into a
single process, enabling data to be stored in the source-read phase to be later
used at the doctree-resolved phase.

What we need is a single process that both reads the source and uses that info
to make the auto markup. With parallel_read_safe enabled, that is only possible
by doing everything in a single Sphinx event (namely, doctree-resolved), and
therefore the read needs to be done manually outside of Sphinx.

Thanks,
Nícolas

>
> Thanks,
>
> jon