linux-kernel - Re: [PATCH v4 10/19] tools/docs: sphinx-build-wrapper: add support to run inside venv

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <4d7acb77be634212056426aee139496da42dc520@intel.com>
Date: Fri, 12 Sep 2025 12:22:42 +0300
From: Jani Nikula <jani.nikula@...ux.intel.com>
To: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
Cc: Jonathan Corbet <corbet@....net>, Linux Doc Mailing List
 <linux-doc@...r.kernel.org>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH v4 10/19] tools/docs: sphinx-build-wrapper: add support
 to run inside venv

On Fri, 12 Sep 2025, Mauro Carvalho Chehab <mchehab+huawei@...nel.org> wrote:
> Em Wed, 10 Sep 2025 13:51:40 +0300
> Jani Nikula <jani.nikula@...ux.intel.com> escreveu:
>
>> On Thu, 04 Sep 2025, Mauro Carvalho Chehab <mchehab+huawei@...nel.org> wrote:
>> > Sometimes, it is desired to run Sphinx from a virtual environment.
>> > Add a command line parameter to automatically build Sphinx from
>> > such environment.  
>> 
>> Why?
>
> In my case, to be able to test build with different Sphinx versions.
> On some distros, only venv works.

I mean why add the complexity of running inside a venv in the wrapper.

>> If you want Sphinx from a virtual environment, you enter the
>> environment, and run the regular build, with sphinx-build from the PATH
>> that points at the venv.
>
> when you do that, ./scripts/spdxcheck.py breaks, affecting checkpatch.

Then you could turn the whole argument around, and say spdxcheck.py
should jump through venv and dependency hoops instead of the docs build.

The point is, it should be the user's responsibility to deal with the
environment and the dependencies.

If they're setting up a virtual environment, and it affects checkpatch,
then they should also include the spdxcheck.py dependencies in the
virtual environment.

This feels like reinventing pipx in a Sphinx wrapper.

We should *reduce* the complexity, not increase it.

>> We don't do this kind of extra magic for any other tools, I honestly
>> don't understand why we'd do this for Sphinx. This just adds complexity
>> for no good reason.
>
>> >
>> > Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
>> > ---
>> >  tools/docs/sphinx-build-wrapper | 30 +++++++++++++++++++++++++++---
>> >  1 file changed, 27 insertions(+), 3 deletions(-)
>> >
>> > diff --git a/tools/docs/sphinx-build-wrapper b/tools/docs/sphinx-build-wrapper
>> > index ea9f8e17b0bc..cf7b30bc40ff 100755
>> > --- a/tools/docs/sphinx-build-wrapper
>> > +++ b/tools/docs/sphinx-build-wrapper
>> > @@ -63,6 +63,7 @@ from jobserver import JobserverExec         # pylint: disable=C0413,C0411,E0401
>> >  #
>> >  #  Some constants
>> >  #
>> > +VENV_DEFAULT = "sphinx_latest"
>> >  MIN_PYTHON_VERSION = PythonVersion("3.7").version
>> >  PAPER = ["", "a4", "letter"]
>> >  
>> > @@ -119,8 +120,9 @@ class SphinxBuilder:
>> >  
>> >          return path
>> >  
>> > -    def __init__(self, builddir, verbose=False, n_jobs=None):
>> > +    def __init__(self, builddir, venv=None, verbose=False, n_jobs=None):
>> >          """Initialize internal variables"""
>> > +        self.venv = venv
>> >          self.verbose = None
>> >  
>> >          #
>> > @@ -195,6 +197,21 @@ class SphinxBuilder:
>> >  
>> >          self.env = os.environ.copy()
>> >  
>> > +        #
>> > +        # If venv command line argument is specified, run Sphinx from venv
>> > +        #
>> > +        if venv:
>> > +            bin_dir = os.path.join(venv, "bin")
>> > +            if not os.path.isfile(os.path.join(bin_dir, "activate")):
>> > +                sys.exit(f"Venv {venv} not found.")
>> > +
>> > +            # "activate" virtual env
>> > +            self.env["PATH"] = bin_dir + ":" + self.env["PATH"]
>> > +            self.env["VIRTUAL_ENV"] = venv
>> > +            if "PYTHONHOME" in self.env:
>> > +                del self.env["PYTHONHOME"]
>> > +            print(f"Setting venv to {venv}")
>> > +
>> >      def run_sphinx(self, sphinx_build, build_args, *args, **pwargs):
>> >          """
>> >          Executes sphinx-build using current python3 command and setting
>> > @@ -209,7 +226,10 @@ class SphinxBuilder:
>> >  
>> >              cmd = []
>> >  
>> > -            cmd.append(sys.executable)
>> > +            if self.venv:
>> > +                cmd.append("python")
>> > +            else:
>> > +                cmd.append(sys.executable)
>> >  
>> >              cmd.append(sphinx_build)
>> >  
>> > @@ -533,11 +553,15 @@ def main():
>> >      parser.add_argument('-j', '--jobs', type=jobs_type,
>> >                          help="Sets number of jobs to use with sphinx-build")
>> >  
>> > +    parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}',
>> > +                        default=None,
>> > +                        help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})')
>> > +
>> >      args = parser.parse_args()
>> >  
>> >      PythonVersion.check_python(MIN_PYTHON_VERSION)
>> >  
>> > -    builder = SphinxBuilder(builddir=args.builddir,
>> > +    builder = SphinxBuilder(builddir=args.builddir, venv=args.venv,
>> >                              verbose=args.verbose, n_jobs=args.jobs)
>> >  
>> >      builder.build(args.target, sphinxdirs=args.sphinxdirs, conf=args.conf,  
>> 
>
>
>
> Thanks,
> Mauro
>

-- 
Jani Nikula, Intel