linux-kernel - Re: Adding CI results to the kernel tree was Re: [RFC v2] drm/msm: Add initial ci/ subdirectory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAF6AEGusO9XAqHNatJLgV+wpVoyyLg1vHtUsnSkAxJeV7n3WNg@mail.gmail.com>
Date:   Wed, 11 May 2022 11:39:56 -0700
From:   Rob Clark <robdclark@...il.com>
To:     Linus Torvalds <torvalds@...ux-foundation.org>
Cc:     Dave Airlie <airlied@...il.com>,
        Tomeu Vizoso <tomeu.vizoso@...labora.com>,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        Maarten Lankhorst <maarten.lankhorst@...ux.intel.com>,
        Maxime Ripard <mripard@...nel.org>,
        Thomas Zimmermann <tzimmermann@...e.de>,
        Daniel Vetter <daniel@...ll.ch>,
        Jonathan Corbet <corbet@....net>, Sean Paul <sean@...rly.run>,
        Abhinav Kumar <quic_abhinavk@...cinc.com>,
        "open list:DOCUMENTATION" <linux-doc@...r.kernel.org>,
        linux-arm-msm <linux-arm-msm@...r.kernel.org>,
        LKML <linux-kernel@...r.kernel.org>,
        dri-devel <dri-devel@...ts.freedesktop.org>,
        freedreno <freedreno@...ts.freedesktop.org>
Subject: Re: Adding CI results to the kernel tree was Re: [RFC v2] drm/msm:
 Add initial ci/ subdirectory

On Wed, May 11, 2022 at 10:33 AM Linus Torvalds
<torvalds@...ux-foundation.org> wrote:
>
> On Tue, May 10, 2022 at 10:07 PM Dave Airlie <airlied@...il.com> wrote:
> >
> > > And use it to store expectations about what the drm/msm driver is
> > > supposed to pass in the IGT test suite.
> >
> > I wanted to loop in Linus/Greg to see if there are any issues raised
> > by adding CI results file to the tree in their minds, or if any other
> > subsystem has done this already, and it's all fine.
> >
> > I think this is a good thing after our Mesa experience, but Mesa has a
> > lot tighter integration here, so I want to get some more opinions
> > outside the group.
>
> Honestly, my immediate reaction is that I think it might be ok, but
>
>  (a) are these things going to absolutely balloon over time?
>
>  (b) should these not be separated out?
>
> Those two issues kind of interact.
>
> If it's a small and targeted test-suite, by all means keep it in the
> kernel, but why not make it part of "tools/testing/selftests"
>
> But if people expect this to balloon and we end up having megabytes of
> test output, then I really think it should be a separate git tree.
>
> A diffstat like this:
>
> >  7 files changed, 791 insertions(+)
>
> is not a problem at all. But I get the feeling that this is just the
> tip of the iceberg, and people will want to not just have the result
> files, but start adding actual *input* files that may be largely
> automated stuff and may be tens of megabytes in size.
>
> Because the result files on their own aren't really self-contained,
> and then people will want to keep them in sync with the test-files
> themselves, and start adding those, and now it *really* is likely very
> unwieldy.

It is missing in this revision of the RFC, but the intention is to
have the gitlab-ci.yml point to a specific commit SHA in the
gfx-ci/drm-ci[1] tree, to solve the problem of keeping the results in
sync with the expectations.  Ie. a kernel commit would control moving
to a new version of i-g-t (and eventually deqp and/or piglit), and at
the same time make any necessary updates in the expectations files.

BR,
-R

[1] https://gitlab.freedesktop.org/gfx-ci/drm-ci

> Or if that doesn't happen, and the actual input test files stay in a
> separate CI repo, and then you end up having random coherency issues
> with that CI repo, and it all gets to be either horribly messy, or the
> result files in the kernel end up really stale.
>
> So honestly, I personally don't see a good end result here.  This
> particular small patch? *This* one looks fine to me, except I really
> think tools/testing/selftests/gpu would be a much more logical place
> for it.
>
> But I don't see a way forward that is sane.
>
> Can somebody argue otherwise?
>
>             Linus