linux-kernel - Re: [PATCH v1 1/1] gitlab-ci: Let project define runner using environment variables

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <ZgWEOJDPl_MEupwz@LeoBras>
Date: Thu, 28 Mar 2024 11:52:40 -0300
From: Leonardo Bras <leobras@...hat.com>
To: Helen Koike <helen.koike@...labora.com>
Cc: Leonardo Bras <leobras@...hat.com>,
	linux-kernel@...r.kernel.org,
	linux-doc@...r.kernel.org
Subject: Re: [PATCH v1 1/1] gitlab-ci: Let project define runner using environment variables

On Thu, Mar 28, 2024 at 10:17:00AM -0300, Helen Koike wrote:
> Hi Leonardo,
> 
> Thanks for the patch.
> 
> On 26/03/2024 22:30, Leonardo Bras wrote:
> > Currently it's not possible to select which runner will handle a pipeline
> > without changing the codebase.
> > 
> > Add CI_TAGS environment variable, which can be used to select a runner
> > either from a commit message, or directly from Gitlab interface.
> > 
> > Also add Documentation for this variable.
> > 
> > Signed-off-by: Leonardo Bras <leobras@...hat.com>
> > ---
> > 
> > This patch goes on top on a previous patchset sent by Helen:
> > https://lore.kernel.org/all/20240228225527.1052240-1-helen.koike@collabora.com/
> > 
> > With this patch I could run CI with gitlab.com runners, by setting
> > CI_TAGS=saas-linux-medium-amd64
> > 
> > The result of this pipeline can be seen in:
> > https://gitlab.com/linux-kernel/linux/-/pipelines/1228999646
> > 
> >   Documentation/ci/gitlab-ci/gitlab-ci.rst | 5 +++++
> >   ci/gitlab-ci/yml/gitlab-ci.yml           | 2 ++
> >   2 files changed, 7 insertions(+)
> > 
> > diff --git a/Documentation/ci/gitlab-ci/gitlab-ci.rst b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > index 4f7ef03cca95..18360da835bd 100644
> > --- a/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > +++ b/Documentation/ci/gitlab-ci/gitlab-ci.rst
> > @@ -304,20 +304,25 @@ Description of Each Variable
> >   **KCI_CHECKPATCH_OPTIONS**
> >       Used in `checkpatch.pl "$KCI_CHECKPATCH_OPTIONS"` (see checkpatch
> >       documentation). It is commonly used with the --ignore flag to suppress
> >       specific warnings generated by checkpatch.pl. It can also be defined in the
> >       commit message, since it is evaluated in run time.
> >   **KCI_PATCH_SERIES_SIZE**
> >       Used to define the size of the patch series, see `job: checkpatch` section
> >       above. It is evaluated in run time, and can be set in the commit message.
> > +**CI_TAGS**
> 
> I would just s/CI/KCI , I'm using KCI_prefix for those that are defined by
> us, since gitlab already uses CI_ prefix.

I was aware of CI_ vs KCI_ prefix, but for some reason at the time I 
decided that since this was about setting the gitlab tags, and not 
something internal to Kernel CI,  it would make more sense to use the CI_ 
prefix. 

But I have no strong opinion on this, so if you think it's better to 
call it KCI_TAGS, let's do that instead.

> 
> We will also need a way to specify tags per job at some point
> KCI_TAGS_$JOB_NAME or something, but this can be added later.

Yeah, agree.

I wish we could specify tags with running requirements for each task in 
some way, and let the gitlab instance to pick the better runner.


> 
> Acked-by: Helen Koike <helen.koike@...labora.com>
> 
> I'll add on top of my tree for now, so we can prepare v2 with this change.
> 
> Thanks
> Helen

Thank you!
Leo

> 
> > +    Used to help choose which runner will deal with the current pipeline.
> > +    If using Gitlab.com runners, set saas-linux-medium-amd64 or a better runner
> > +    so there is enough resources to build & commit the base image.
> > +
> >   .. _triggering-pipelines-from-command-line:
> >   Triggering Pipelines from Command Line
> >   --------------------------------------
> >   Pipelines can be triggered from the command line with custom variables using the
> >   `GitLab CLI tool <https://docs.gitlab.com/ee/editor_extensions/gitlab_cli>`_.
> >   Example:
> > diff --git a/ci/gitlab-ci/yml/gitlab-ci.yml b/ci/gitlab-ci/yml/gitlab-ci.yml
> > index 57b9c0290471..359b7715e3ab 100644
> > --- a/ci/gitlab-ci/yml/gitlab-ci.yml
> > +++ b/ci/gitlab-ci/yml/gitlab-ci.yml
> > @@ -33,20 +33,22 @@ workflow:
> >       - if: $FORCE_CI == 'true'
> >   variables:
> >     FDO_UPSTREAM_REPO: helen.fornazier/linux   # The repo where to look for cached images
> >       # ccache builds in gitlab-runner to speed up builds
> >     SMATCH_DB_DIR: /smatch/smatch_data
> >     # exit code of bash script on `script` will be the exit code of the job
> >     FF_USE_NEW_BASH_EVAL_STRATEGY: "true"
> >   default:
> > +  tags:
> > +    - $CI_TAGS
> >     artifacts:
> >       paths:
> >         - artifacts/
> >       when: always
> >   include:
> >     - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/ci-fairy.yml'
> >     - remote: 'https://gitlab.freedesktop.org/freedesktop/ci-templates/-/raw/16bc29078de5e0a067ff84a1a199a3760d3b3811/templates/debian.yml'
> >     - ci/gitlab-ci/yml/kernel-combinations.yml
>