linux-kernel - Re: [RFC PATCH] drm/ci: add helper script update-xfails.py

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <c25165891e0628b3a3a50cc528a86b2b2a65b7ae.camel@collabora.com>
Date:   Thu, 28 Sep 2023 09:10:58 +0200
From:   Sergi Blanch Torne <sergi.blanch.torne@...labora.com>
To:     Helen Koike <helen.koike@...labora.com>,
        dri-devel@...ts.freedesktop.org
Cc:     airlied@...il.com, daniel@...ll.ch, linux-kernel@...r.kernel.org,
        robdclark@...il.com, daniel@...ishbar.org,
        vignesh.raman@...labora.com, jani.nikula@...ux.intel.com,
        mripard@...nel.org, michel.daenzer@...lbox.org,
        quic_abhinavk@...cinc.com, quic_jesszhan@...cinc.com,
        david.heidelberg@...labora.com, guilherme.gallo@...labora.com
Subject: Re: [RFC PATCH] drm/ci: add helper script update-xfails.py

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi Helen,

On Wed, 2023-09-27 at 19:28 -0300, Helen Koike wrote:
> > > +def get_unit_test_name_and_results(unit_test):
> > > +    if "Artifact results/failures.csv not found" in unit_test:
> > > +        return None, None
> > > +    unit_test_name, unit_test_result =
> > > unit_test.strip().split(",")
> > > +    return unit_test_name, unit_test_result
> > 
> > Suggestion: it is not managing empty lines or comments. By now,
> > there
> > aren't, but they could be found.
> 
> Indeed.

Just add a new if statement to discard if the strings start with # or
strip the line and check the length. Perhaps we can think of other
assertions to sanitise the string.

> > Suggestion: Sometimes tests fails with different status ("Fail" to
> > "Crash" for example) and the expectations should be updated with
> > the
> > newer status.
> 
> The status is only present in the fails and not in the flakes list,
> so I 
> update it with add_unit_test_or_update_result_to_fails_if_present() 
> function below, make sense?

Absolutely, sorry that I didn't see this was a process included in the
last if statement. If it is present in the fails' file (that includes
the test name and its state) you do exactly what's necessary: add if
not present, update if it was already in the file.

> 
Regards,


- -- 
Sergi Blanch Torné
Senior Software Engineer

Collabora Ltd.
Platinum Building, St John's Innovation Park, Cambridge CB4 0DS, UK
Registered in England & Wales, no. 5513718

-----BEGIN PGP SIGNATURE-----

iHUEARYIAB0WIQQwWRK68l+taJfhwqAto5bHyTm9RwUCZRUnAgAKCRAto5bHyTm9
R53NAP9T2OCiwbnEjv+H0CQg/eK1xGe7yS/3cqjaPFRvvZPp1wD/V1H9NuhpRR6M
8+QZgbsS/swSPdwYABtcz+75CKpuJwo=
=XRRO
-----END PGP SIGNATURE-----