linux-kernel - Re: [PATCH] dt-bindings: Improve validation build error handling

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAK7LNASNugzCjzgObh8+fMvDhUNspapZy9rCA57eX3cLs6NW_g@mail.gmail.com>
Date:   Fri, 15 Nov 2019 14:58:45 +0900
From:   Masahiro Yamada <yamada.masahiro@...ionext.com>
To:     Rob Herring <robh@...nel.org>
Cc:     Jeffrey Hugo <jhugo@...eaurora.org>,
        DTML <devicetree@...r.kernel.org>,
        "linux-kernel@...r.kernel.org" <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] dt-bindings: Improve validation build error handling

On Fri, Nov 15, 2019 at 1:34 AM Rob Herring <robh@...nel.org> wrote:
>
> On Thu, Nov 14, 2019 at 9:21 AM Jeffrey Hugo <jhugo@...eaurora.org> wrote:
> >
> > On 11/13/2019 2:05 PM, Rob Herring wrote:
> > > Schema errors can cause make to exit before useful information is
> > > printed. This leaves developers wondering what's wrong. It can be
> > > overcome passing '-k' to make, but that's not an obvious solution.
> > > There's 2 scenarios where this happens.
> > >
> > > When using DT_SCHEMA_FILES to validate with a single schema, any error
> > > in the schema results in processed-schema.yaml being empty causing a
> > > make error. The result is the specific errors in the schema are never
> > > shown because processed-schema.yaml is the first target built. Simply
> > > making processed-schema.yaml last in extra-y ensures the full schema
> > > validation with detailed error messages happen first.
> > >
> > > The 2nd problem is while schema errors are ignored for
> > > processed-schema.yaml, full validation of the schema still runs in
> > > parallel and any schema validation errors will still stop the build when
> > > running validation of dts files. The fix is to not add the schema
> > > examples to extra-y in this case. This means 'dtbs_check' is no longer a
> > > superset of 'dt_binding_check'. Update the documentation to make this
> > > clear.
> > >
> > > Cc: Jeffrey Hugo <jhugo@...eaurora.org>
> > > Cc: Masahiro Yamada <yamada.masahiro@...ionext.com>
> > > Signed-off-by: Rob Herring <robh@...nel.org>
> >
> > I injected a syntax error into a random binding file, and compared the
> > output with and without this patch.  This patch makes a massive
> > improvement in giving the user the necessary information to identify and
> > fix issues.  Thanks!

Could you show me an example of the injected syntax error,
and how this commit will improve the diagnostic?

For example, I changed as follows:

diff --git a/Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml
b/Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml
index 47bc1ac36426..358cb1fa4bb6 100644
--- a/Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml
+++ b/Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml
@@ -89,7 +89,7 @@ required:
   - clocks

 allOf:
-  - if:
+  - if2:
       properties:
         compatible:
           contains:

The tool clearly explains the cause of the error.
I am struggling to understand what the current problem is.

masahiro@pug:~/ref/linux$ make -j8  ARCH=arm  dt_binding_check
  CHKDT   Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml
  DTC     Documentation/devicetree/bindings/arm/altera.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/nxp/lpc32xx.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/pmu.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/rockchip.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/primecell.example.dt.yaml
  DTC     Documentation/devicetree/bindings/arm/renesas.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/nxp/lpc32xx.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/pmu.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/altera.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/rockchip.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/altera/socfpga-clk-manager.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/primecell.example.dt.yaml
  CHECK   Documentation/devicetree/bindings/arm/renesas.example.dt.yaml
/home/masahiro/ref/linux/Documentation/devicetree/bindings/gpu/arm,mali-midgard.yaml:
allOf:0: 'if2' is not one of ['$ref', 'if', 'then', 'else']
make[1]: *** [Documentation/devicetree/bindings/Makefile;12:
Documentation/devicetree/bindings/gpu/arm,mali-midgard.example.dts]
Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile;1266: dt_binding_check] Error 2

> BTW, update dtschema and you'll get better (or more at least) messages
> when 'is not valid under any of the given schemas' errors occur.
>
> > Tested-by: Jeffrey Hugo <jhugo@...eaurora.org>
>
> Thanks.
>
> Rob

-- 
Best Regards
Masahiro Yamada