Message-ID: <20250910105454.6a6c57e6@foz.lan>
Date: Wed, 10 Sep 2025 10:54:54 +0200
From: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>
To: Randy Dunlap <rdunlap@...radead.org>
Cc: Linux Doc Mailing List <linux-doc@...r.kernel.org>, Jonathan Corbet
 <corbet@....net>, linux-kernel@...r.kernel.org
Subject: Re: [PATCH] kernel-doc: add support for handling global variables

Em Tue, 9 Sep 2025 23:13:24 -0700
Randy Dunlap <rdunlap@...radead.org> escreveu:

> On 9/9/25 10:59 PM, Randy Dunlap wrote:
> > Hi Mauro,
> > 
> > On 9/9/25 9:23 PM, Mauro Carvalho Chehab wrote:  
> >> Em Tue, 9 Sep 2025 17:02:00 -0700
> >> Randy Dunlap <rdunlap@...radead.org> escreveu:
> >>  
> >>> On 9/9/25 4:50 PM, Randy Dunlap wrote:  
> >>>>
> >>>>
> >>>> On 9/9/25 4:49 PM, Randy Dunlap wrote:    
> >>>>>
> >>>>>
> >>>>> On 9/9/25 4:09 PM, Mauro Carvalho Chehab wrote:    
> >>>>>> Em Tue, 9 Sep 2025 14:06:43 -0700
> >>>>>> Randy Dunlap <rdunlap@...radead.org> escreveu:
> >>>>>>    
> >>>>>>> On 9/9/25 12:58 PM, Mauro Carvalho Chehab wrote:    
> >>>>>>>> Em Tue, 9 Sep 2025 00:27:20 -0700
> >>>>>>>> Randy Dunlap <rdunlap@...radead.org> escreveu:    
> >>>>>>    
> >>>>>>>>> +.. kernel-doc:: init/kdoc-globals-test.c
> >>>>>>>>> +   :identifiers:
> >>>>>>>>>
> >>>>>>>>> The html output says
> >>>>>>>>> "Kernel Globals"
> >>>>>>>>> but nothing else.      
> >>>>>>>>
> >>>>>>>> I usually don't add :identifiers: on kernel-doc entries. If you use
> >>>>>>>> identifiers, you need to explicitly tell what symbols you want.      
> >>>>>>>
> >>>>>>> Well, it worked/works without using having any identifiers listed, and
> >>>>>>> the docs in Documentation/doc-guide/kernel-doc.rst says that they are
> >>>>>>> optional:
> >>>>>>>
> >>>>>>> identifiers: *[ function/type ...]*
> >>>>>>>   Include documentation for each *function* and *type* in *source*.
> >>>>>>>   If no *function* is specified, the documentation for all functions
> >>>>>>>   and types in the *source* will be included.
> >>>>>>>   *type* can be a struct, union, enum, or typedef identifier.    
> >>>>>>  
> 
> 
> 
> >>>>> Anyway, does this take away something that currently works?    
> >>>
> >>> The output looks the same with this patch AFAICT.  
> >>
> >> run it in verbose mode to see what command line was passed to
> >> the file:
> >>
> >> 	$ make SPHINXDIRS=your_test_dir V=1 htmldocs
> >>
> >> This should be printing how the kernel-doc.py command line would be(*):
> >>
> >> 	scripts/kernel-doc.py -rst -enable-lineno ./include/linux/peci.h
> >> 	./include/linux/peci.h:20 Scanning doc for struct peci_controller_ops
> >> 	./include/linux/peci.h:32 Scanning doc for struct peci_controller
> >> 	./include/linux/peci.h:58 Scanning doc for struct peci_device
> >> 	./include/linux/peci.h:88 Scanning doc for struct peci_request
> >>
> >> (*) the kerneldoc.py extension doesn't call kernel-doc.py, but instead
> >>     run directly the Python classes from the library. Yet, to help one
> >>     to debug it, the command line is displayed.  
> > 
> > I see. Thanks.
> > 
> > I get this if I list all of them (on 2 separate identifiers lines):
> > 
> > ../scripts/kernel-doc.py -rst -enable-lineno -function ROOT_DEV -function system_state -function saved_command_line -function diskseq ../init/kdoc-globals-test.c
> > ../init/kdoc-globals-test.c:5 Scanning doc for global ROOT_DEV
> > ../init/kdoc-globals-test.c:15 Scanning doc for global system_state
> > ../init/kdoc-globals-test.c:27 Scanning doc for global saved_command_line
> > ../init/kdoc-globals-test.c:33 Scanning doc for global loops_per_jiffy
> > ../init/kdoc-globals-test.c:40 Scanning doc for global preset_lpj
> > ../init/kdoc-globals-test.c:49 Scanning doc for global linux_proc_banner
> > ../init/kdoc-globals-test.c:63 Scanning doc for global linux_banner
> > ../init/kdoc-globals-test.c:72 Scanning doc for global diskseq
> > ../init/kdoc-globals-test.c:80 Scanning doc for global rtnl_mutex
> > ../scripts/kernel-doc.py -rst -enable-lineno -function loops_per_jiffy -function preset_lpj -function linux_proc_banner -function linux_banner ../init/kdoc-globals-test.c
> > 
> > or this is I don't use the identifiers line at all:
> > 
> > ../scripts/kernel-doc.py -rst -enable-lineno ../init/kdoc-globals-test.c
> > ../init/kdoc-globals-test.c:5 Scanning doc for global ROOT_DEV
> > ../init/kdoc-globals-test.c:15 Scanning doc for global system_state
> > ../init/kdoc-globals-test.c:27 Scanning doc for global saved_command_line
> > ../init/kdoc-globals-test.c:33 Scanning doc for global loops_per_jiffy
> > ../init/kdoc-globals-test.c:40 Scanning doc for global preset_lpj
> > ../init/kdoc-globals-test.c:49 Scanning doc for global linux_proc_banner
> > ../init/kdoc-globals-test.c:63 Scanning doc for global linux_banner
> > ../init/kdoc-globals-test.c:72 Scanning doc for global diskseq
> > ../init/kdoc-globals-test.c:80 Scanning doc for global rtnl_mutex
> > 
> > 
> > And then both of them report these warnings (already discussed):
> > 
> > Documentation/core-api/kernel-api:435: ../init/kdoc-globals-test.c:10: WARNING: Invalid C declaration: Expected end of definition. [error at 32]
> >   enum system_states system_state __read_mostly;
> >   --------------------------------^
> > Documentation/core-api/kernel-api:435: ../init/kdoc-globals-test.c:20: WARNING: Invalid C declaration: Expected end of definition. [error at 25]
> >   char *saved_command_line __ro_after_init;
> >   -------------------------^
> > 
> > and the 3 globals with initialization values are skipped/omitted.
> > 
> > So to get "all identifiers," I should just omit the :identifiers:
> > line completely. kernel-doc.rst could use some clarification on that
> > point.  
> 
> 
> Oh darn, the html output is different:
> 
> when I omit the :identifiers: line, I see:
> 
> Kernel Globals
> dev_t ROOT_DEV;
> system root device
> 
> enum system_states system_state __read_mostly;
> system state used during boot or suspend/hibernate/resume
> 
> char *saved_command_line __ro_after_init;
> kernel’s command line, saved from use at any later time in the kernel.
> 
> unsigned long preset_lpj;
> lpj (loops per jiffy) value set from kernel command line using “lpj=VALUE”
> 
> static atomic64_t diskseq;
> unique sequence number for block device instances
> 
> and when I list all 8 identifiers (on 2 separate lines), I see:
> 
> Kernel Globals
> dev_t ROOT_DEV;
> system root device
> 
> static atomic64_t diskseq;
> unique sequence number for block device instances
> 
> unsigned long preset_lpj;
> lpj (loops per jiffy) value set from kernel command line using “lpj=VALUE”
> 
> so for some reason, system_state, saved_command_line, and diskseq are
> skipped/omitted when I list all 8 identifiers.

It is because of the regex we're using:

+        VAR_ATTRIBS = [
+            "extern",
+        ]
+        OPTIONAL_VAR_ATTR = "^(?:" + "|".join(VAR_ATTRIBS) + ")?"
+
+        r= KernRe(OPTIONAL_VAR_ATTR + r"(\w.*)\s+([\w_]+)[\d\]\[]*\s*;(?:#.*)?$")
...
+        declaration_name = r.group(2)
+	 var_type = r.group(0)

the above stores the entire line as var_type, and the last string
as declaration_name. The output logic will use var_type as function
prototype, and declaration_name as the key to check for identifiers.

Here:

	enum system_states system_state __read_mostly;
	char *saved_command_line __ro_after_init;

the regex would pick __read_mostly and __ro_after_init as 
identifiers.

Once we fix the regex to pick the actual var name, both ways should work
equally.

Btw, the best way to test complex regular expressions is, IMHO, to use:

	https://regex101.com/

Don't forget to select first <Python> as the default is pcre2.

there, you can place the regex on the top and several test strings and
see if everything is working.

Anyway, I did some changes at the code for it to better parse your
examples. I'm also storing two additional fields captured at parsing
time:

1. full_proto: the prototype before any transformation;
2. default_val: if not None, contains the initialized value.

Right now, I'm using only default_val ato output, but, with both,
we can play with printing them or not at kdoc_output logic.

Thanks,
Mauro

[PATCH v2] kernel-doc: add support for handling global variables

Specially on kAPI, sometimes it is desirable to be able to
describe global variables that are part of kAPI.

Documenting vars with Sphinx is simple, as we don't need
to parse a data struct. All we need is the variable
declaration and use natice C domain ::c:var: to format it
for us.

Add support for it.

Link: https://lore.kernel.org/linux-doc/491c3022-cef8-4860-a945-c9c4a3b63c09@infradead.org/T/#m947c25d95cb1d96a394410ab1131dc8e9e5013f1
Suggested-by: Randy Dunlap <rdunlap@...radead.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@...nel.org>

diff --git a/scripts/lib/kdoc/kdoc_output.py b/scripts/lib/kdoc/kdoc_output.py
index 1eca9a918558..469f01a74724 100644
--- a/scripts/lib/kdoc/kdoc_output.py
+++ b/scripts/lib/kdoc/kdoc_output.py
@@ -199,6 +199,10 @@ class OutputFormat:
             self.out_enum(fname, name, args)
             return self.data
 
+        if dtype == "global":
+            self.out_global(fname, name, args)
+            return self.data
+
         if dtype == "typedef":
             self.out_typedef(fname, name, args)
             return self.data
@@ -227,6 +231,9 @@ class OutputFormat:
     def out_enum(self, fname, name, args):
         """Outputs an enum"""
 
+    def out_global(self, fname, name, args):
+        """Outputs a global variable"""
+
     def out_typedef(self, fname, name, args):
         """Outputs a typedef"""
 
@@ -472,6 +479,24 @@ class RestFormat(OutputFormat):
         self.lineprefix = oldprefix
         self.out_section(args)
 
+    def out_global(self, fname, name, args):
+        oldprefix = self.lineprefix
+        ln = args.declaration_start_line
+        prototype = args.other_stuff["var_type"]
+
+        self.data += f"\n\n.. c:var:: {prototype}\n\n"
+
+        self.print_lineno(ln)
+        self.lineprefix = "  "
+        self.output_highlight(args.get('purpose', ''))
+        self.data += "\n"
+
+        if args.other_stuff["default_val"]:
+            self.data += f'{self.lineprefix}**Initialization**\n\n'
+            self.output_highlight(f'default: ``{args.other_stuff["default_val"]}``')
+
+        self.out_section(args)
+
     def out_typedef(self, fname, name, args):
 
         oldprefix = self.lineprefix
@@ -772,6 +797,26 @@ class ManFormat(OutputFormat):
             self.data += f'.SH "{section}"' + "\n"
             self.output_highlight(text)
 
+    def out_global(self, fname, name, args):
+        out_name = self.arg_name(args, name)
+        prototype = args.other_stuff["var_type"]
+
+        self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
+
+        self.data += ".SH NAME\n"
+        self.data += f"{prototype} \\- {args['purpose']}\n"
+
+        self.data += ".SH SYNOPSIS\n"
+        self.data += f"enum {name}" + " {\n"
+
+        if args.other_stuff["default_val"]:
+            self.data += f'.SH "Initialization"' + "\n"
+            self.output_highlight(f'default: {args.other_stuff["default_val"]}')
+
+        for section, text in args.sections.items():
+            self.data += f'.SH "{section}"' + "\n"
+            self.output_highlight(text)
+
     def out_typedef(self, fname, name, args):
         module = self.modulename
         purpose = args.get('purpose')
diff --git a/scripts/lib/kdoc/kdoc_parser.py b/scripts/lib/kdoc/kdoc_parser.py
index 574972e1f741..93a2fcebec28 100644
--- a/scripts/lib/kdoc/kdoc_parser.py
+++ b/scripts/lib/kdoc/kdoc_parser.py
@@ -64,7 +64,7 @@ type_param = KernRe(r"@(\w*((\.\w+)|(->\w+))*(\.\.\.)?)", cache=False)
 # Tests for the beginning of a kerneldoc block in its various forms.
 #
 doc_block = doc_com + KernRe(r'DOC:\s*(.*)?', cache=False)
-doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef)\b\s*(\w*)", cache = False)
+doc_begin_data = KernRe(r"^\s*\*?\s*(struct|union|enum|typedef|global)\b\s*(\w*)", cache = False)
 doc_begin_func = KernRe(str(doc_com) +			# initial " * '
                         r"(?:\w+\s*\*\s*)?" + 		# type (not captured)
                         r'(?:define\s+)?' + 		# possible "define" (not captured)
@@ -886,6 +886,57 @@ class KernelDoc:
         self.output_declaration('enum', declaration_name,
                                 purpose=self.entry.declaration_purpose)
 
+    def dump_global(self, ln, proto):
+        """
+        Stores global variables that are part of kAPI.
+        """
+        VAR_ATTRIBS = [
+            "extern",
+        ]
+        OPTIONAL_VAR_ATTR = "^(?:" + "|".join(VAR_ATTRIBS) + ")?"
+
+        sub_prefixes = [
+            (KernRe(r"__read_mostly"), ""),
+            (KernRe(r"__ro_after_init"), ""),
+            (KernRe(r"(?://.*)$"), ""),
+            (KernRe(r"(?:/\*.*\*/)"), ""),
+            (KernRe(r";$"), ""),
+        ]
+
+        #
+        # Store the full prototype before modifying it
+        #
+        full_proto = proto
+
+        #
+        # Drop comments and macros to have a pure C prototype
+        #
+        for search, sub in sub_prefixes:
+            proto = search.sub(sub, proto)
+
+        proto = proto.rstrip()
+
+        #
+        # Variable name is at the end of the declaration
+        #
+
+        r= KernRe(OPTIONAL_VAR_ATTR + r"\w.*\s+([\w_]+)\s*[\d\]\[]*\s*(=.*)?")
+        if not r.match(proto):
+           self.emit_msg(ln,f"{proto}: can't parse variable")
+           return
+
+        var_type = r.group(0)
+        declaration_name = r.group(1)
+        default_val = r.group(2)
+        if default_val:
+            default_val = default_val.lstrip("=").strip()
+
+        self.output_declaration("global", declaration_name,
+                                full_proto=full_proto,
+                                var_type=var_type,
+                                default_val=default_val,
+                                purpose=self.entry.declaration_purpose)
+
     def dump_declaration(self, ln, prototype):
         """
         Stores a data declaration inside self.entries array.
@@ -897,6 +948,8 @@ class KernelDoc:
             self.dump_typedef(ln, prototype)
         elif self.entry.decl_type in ["union", "struct"]:
             self.dump_struct(ln, prototype)
+        elif self.entry.decl_type == "global":
+            self.dump_global(ln, prototype)
         else:
             # This would be a bug
             self.emit_message(ln, f'Unknown declaration type: {self.entry.decl_type}')

Powered by blists - more mailing lists