• 概要
  • Pipeline types
    • Docs-only MR pipeline
    • Code-only MR pipeline
    • Frontend-only MR pipeline
    • QA-only MR pipeline
  • workflow:rules
  • PostgreSQL versions testing
    • Current versions testing
    • Long-term plan
  • Test jobs
  • Review app jobs
  • As-if-FOSS jobs
• パフォーマンス
  • Interruptible pipelines
  • Caching strategy
  • Pre-clone step
• CI configuration internals
  • ステージ
  • Default image
  • Default variables
  • Common job definitions
  • rules, if: conditions and changes: patterns
    • if: conditions
    • changes: patterns

Pipelines for the GitLab project

Pipelines for https://gitlab.com/gitlab-org/gitlab and https://gitlab.com/gitlab-org/gitlab-foss (as well as the dev instance’s mirrors) are configured in the usual .gitlab-ci.yml which itself includes files under .gitlab/ci/ for easier maintenance.

We’re striving to dogfood GitLab CI/CD features and best-practices as much as possible.

概要

Pipeline types

Since we use the rules: and needs: keywords extensively, we have four main pipeline types which are described below. Note that an MR that includes multiple types of changes would have a pipelines that include jobs from multiple types (e.g. a combination of docs-only and code-only pipelines).

Docs-only MR pipeline

Reference pipeline: https://gitlab.com/gitlab-org/gitlab/pipelines/135236627

Code-only MR pipeline

Reference pipeline: https://gitlab.com/gitlab-org/gitlab/pipelines/136295694

Frontend-only MR pipeline

Reference pipeline: https://gitlab.com/gitlab-org/gitlab/pipelines/134661039

QA-only MR pipeline

Reference pipeline: https://gitlab.com/gitlab-org/gitlab/pipelines/134645109

workflow:rules

We’re using the workflow:rules keyword to define default rules to determine whether or not a pipeline is created.

These rules are defined in https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml and are as follows:

1. If $FORCE_GITLAB_CI is set, create a pipeline.
2. For merge requests, create a pipeline.
3. For master branch, create a pipeline (this includes on schedules, pushes, merges, etc.).
4. For tags, create a pipeline.
5. If $GITLAB_INTERNAL isn’t set, don’t create a pipeline.
6. For stable, auto-deploy, and security branches, create a pipeline.
7. For any other cases (e.g. when pushing a branch with no MR for it), no pipeline is created.

PostgreSQL versions testing

Current versions testing

Long-term plan

We follow the PostgreSQL versions shipped with Omnibus GitLab:

Test jobs

Consult GitLab tests in the Continuous Integration (CI) context for more information.

Review app jobs

Consult the Review Apps dedicated page for more information.

As-if-FOSS jobs

The * as-if-foss jobs allows to run GitLab’s test suite “as-if-FOSS”, meaning as if the jobs would run in the context of the gitlab-org/gitlab-foss project. These jobs are only created in the following cases:

• gitlab-org/security/gitlab merge requests.
• Merge requests which include RUN AS-IF-FOSS in their title.
• Merge requests that changes the CI configuration.

The * as-if-foss jobs have the FOSS_ONLY='1' variable set and gets their EE-specific folders removed before the tests start running.

The intent is to ensure that a change won’t introduce a failure once the gitlab-org/gitlab project will be synced to the gitlab-org/gitlab-foss project.

パフォーマンス

Interruptible pipelines

By default, all jobs are interruptible, except the dont-interrupt-me job which runs automatically on master, and is manual otherwise.

If you want a running pipeline to finish even if you push new commits to a merge request, be sure to start the dont-interrupt-me job before pushing.

Caching strategy

1. All jobs must only pull caches by default.
2. All jobs must be able to pass with an empty cache. In other words, caches are only there to speed up jobs.
3. We currently have 6 different caches defined in .gitlab/ci/global.gitlab-ci.yml, with fixed keys:
   • .rails-cache.
   • .static-analysis-cache.
   • .qa-cache
   • .yarn-cache.
   • .assets-compile-cache (the key includes ${NODE_ENV} so it’s actually two different caches).
4. Only 6 specific jobs, running in 2-hourly scheduled pipelines, are pushing (i.e. updating) to the caches:
   • update-rails-cache, defined in .gitlab/ci/rails.gitlab-ci.yml.
   • update-static-analysis-cache, defined in .gitlab/ci/rails.gitlab-ci.yml.
   • update-qa-cache, defined in .gitlab/ci/qa.gitlab-ci.yml.
   • update-assets-compile-production-cache, defined in .gitlab/ci/frontend.gitlab-ci.yml.
   • update-assets-compile-test-cache, defined in .gitlab/ci/frontend.gitlab-ci.yml.
   • update-yarn-cache, defined in .gitlab/ci/frontend.gitlab-ci.yml.
5. These jobs will run in merge requests whose title include UPDATE CACHE.

Pre-clone step

The gitlab-org/gitlab project on GitLab.com uses a pre-clone step to seed the project with a recent archive of the repository. This is done for several reasons:

• It speeds up builds because a 800 MB download only takes seconds, as opposed to a full Git clone.
• It significantly reduces load on the file server, as smaller deltas mean less time spent in git pack-objects.

The pre-clone step works by using the CI_PRE_CLONE_SCRIPT variable defined by GitLab.com shared runners.

The CI_PRE_CLONE_SCRIPT is currently defined as a project CI/CD variable:

echo "Downloading archived master..."
wget -O /tmp/gitlab.tar.gz https://storage.googleapis.com/gitlab-ci-git-repo-cache/project-278964/gitlab-master.tar.gz

if [ ! -f /tmp/gitlab.tar.gz ]; then
    echo "Repository cache not available, cloning a new directory..."
    exit
fi

rm -rf $CI_PROJECT_DIR
echo "Extracting tarball into $CI_PROJECT_DIR..."
mkdir -p $CI_PROJECT_DIR
cd $CI_PROJECT_DIR
tar xzf /tmp/gitlab.tar.gz
rm -f /tmp/gitlab.tar.gz
chmod a+w $CI_PROJECT_DIR

The first step of the script downloads gitlab-master.tar.gz from Google Cloud Storage. There is a GitLab CI job named cache-repo that is responsible for keeping that archive up-to-date. Every two hours on a scheduled pipeline, it does the following:

1. Creates a fresh clone of the gitlab-org/gitlab repository on GitLab.com.
2. Saves the data as a .tar.gz.
3. Uploads it into the Google Cloud Storage bucket.

When a CI job runs with this configuration, you’ll see something like this:

$ eval "$CI_PRE_CLONE_SCRIPT"
Downloading archived master...
Extracting tarball into /builds/group/project...
Fetching changes...
Reinitialized existing Git repository in /builds/group/project/.git/

Note that the Reinitialized existing Git repository message shows that the pre-clone step worked. The runner runs git init, which overwrites the Git configuration with the appropriate settings to fetch from the GitLab repository.

CI_REPO_CACHE_CREDENTIALS contains the Google Cloud service account JSON for uploading to the gitlab-ci-git-repo-cache bucket. These credentials are stored in the 1Password GitLab.com Production vault.

Note that this bucket should be located in the same continent as the runner, or network egress charges will apply.

CI configuration internals

ステージ

The current stages are:

• sync: This stage is used to synchronize changes from https://gitlab.com/gitlab-org/gitlab to https://gitlab.com/gitlab-org/gitlab-foss.
• prepare: This stage includes jobs that prepare artifacts that are needed by jobs in subsequent stages.
• build-images: This stage includes jobs that prepare Docker images that are needed by jobs in subsequent stages or downstream pipelines.
• fixtures: This stage includes jobs that prepare fixtures needed by frontend tests.
• test: This stage includes most of the tests, DB/migration jobs, and static analysis jobs.
• post-test: This stage includes jobs that build reports or gather data from the test stage’s jobs (e.g. coverage, Knapsack metadata etc.).
• review-prepare: This stage includes a job that build the CNG images that are later used by the (Helm) Review App deployment (see Review Apps for details).
• review: This stage includes jobs that deploy the GitLab and Docs Review Apps.
• qa: This stage includes jobs that perform QA tasks against the Review App that is deployed in the previous stage.
• post-qa: This stage includes jobs that build reports or gather data from the qa stage’s jobs (e.g. Review App performance report).
• pages: This stage includes a job that deploys the various reports as GitLab Pages (e.g. coverage-ruby, coverage-javascript, webpack-report.

Default image

The default image is defined in .gitlab-ci.yml.

It includes Ruby, Go, Git, Git LFS, Chrome, Node, Yarn, PostgreSQL, and Graphics Magick.

The images used in our pipelines are configured in the gitlab-org/gitlab-build-images project, which is push-mirrored to gitlab/gitlab-build-images for redundancy.

The current version of the build images can be found in the “Used by GitLab section”.

Default variables

In addition to the predefined variables, each pipeline includes default variables defined in https://gitlab.com/gitlab-org/gitlab/blob/master/.gitlab-ci.yml.

Common job definitions

Most of the jobs extend from a few CI definitions defined in .gitlab/ci/global.gitlab-ci.yml that are scoped to a single configuration parameter.

rules, if: conditions and changes: patterns

We’re using the rules keyword extensively.

All rules definitions are defined in https://gitlab.com/gitlab-org/gitlab/-/blob/master/.gitlab/ci/rules.gitlab-ci.yml, then included in individual jobs via extends.

The rules definitions are composed of if: conditions and changes: patterns, which are also defined in rules.gitlab-ci.yml and included in rules definitions via YAML anchors

if: conditions

changes: patterns

Return to Development documentation