GitLab CI Reference

Anatomy of .gitlab-ci.yml

A pipeline is created from .gitlab-ci.yml. It contains jobs. Each job belongs to a stage. Jobs in the same stage run in parallel, and the next stage starts after the jobs of the previous stage succeed. If you do not define stages, the defaults are .pre, build, test, deploy and .post, and a job with no stage goes to test.

stages:
  - build
  - test
  - deploy

default:
  image: alpine:3.20
  before_script:
    - echo "Running on $CI_COMMIT_REF_NAME"

variables:
  APP_ENV: ci

build:
  stage: build
  script:
    - mkdir -p dist
    - echo "$APP_ENV" > dist/app.txt
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

test:
  stage: test
  script:
    - test -f dist/app.txt

What each keyword does:

• script: the shell commands the runner executes. image: the container image the job runs in.
• before_script: runs before script, after artifacts are restored, and is concatenated with script into one shell. after_script runs in a separate new shell and does not change the job exit code.
• default: each default keyword is copied to every job that does not already define it. A job that sets its own before_script does not also get the default one.
• variables: CI/CD variables for all jobs (top level) or for one job. Job-level values override the top-level ones.
• Names starting with a dot (.setup) are hidden jobs. They are never run and serve as templates.

For a first hands-on pipeline, follow the first GitLab CI pipeline tutorial on this site.

Rules and workflow:rules

rules decide whether a job is created. Rules are evaluated in order until the first match. If no rule matches, the job is not added to the pipeline. rules is the documented way to control jobs: the GitLab reference lists only and except as deprecated keywords that are no longer recommended.

• if: a CI/CD variable expression. Common variables are $CI_PIPELINE_SOURCE, $CI_COMMIT_BRANCH, $CI_COMMIT_TAG and $CI_DEFAULT_BRANCH.
• changes: match when listed files changed (paths, optionally compare_to).
• when: on_success (default), manual, delayed, never and others. allow_failure: whether the pipeline continues if the job fails (default false, but see manual jobs below).

test:
  stage: test
  script:
    - make test
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

docs:
  stage: build
  script:
    - make docs
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
      changes:
        paths:
          - 'docs/**/*'

release:
  stage: deploy
  script:
    - make release
  rules:
    - if: $CI_COMMIT_TAG

These three jobs cover the standard patterns: merge request pipelines, the default branch, and tags. $CI_COMMIT_BRANCH is not available in merge request pipelines or tag pipelines, so the tag job tests $CI_COMMIT_TAG.

workflow:rules: which pipelines exist

workflow:rules controls whether a pipeline is created at all. Without it, one push to a branch with an open merge request can create both a branch pipeline and a merge request pipeline. This is the pattern from the documentation, plus a tag rule:

workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS
      when: never
    - if: $CI_COMMIT_BRANCH
    - if: $CI_COMMIT_TAG

Read it top to bottom. Merge request events run. A branch push with an open merge request is skipped (when: never), so no duplicate. Other branch pushes run, and tags run. $CI_COMMIT_BRANCH is not available in tag pipelines, so the last rule is what lets them through. If no workflow rule matches, the pipeline does not run, and a job configured for tags never runs if the workflow prevents tag pipelines. Check that every pipeline type you need matches a rule.

Artifacts vs cache

Both define their paths relative to the project directory and cannot link to files outside it. Their purpose differs.

.deps-cache:
  cache:
    key:
      files:
        - package-lock.json
    paths:
      - .deps/

build:
  extends: .deps-cache
  stage: build
  script:
    - make deps build
  artifacts:
    paths:
      - dist/
    expire_in: 1 week

test:
  extends: .deps-cache
  stage: test
  cache:
    policy: pull
  script:
    - make test
  artifacts:
    when: always
    paths:
      - reports/
    expire_in: 1 week

• artifacts:paths lists what to keep. expire_in sets how long. If you omit it, the instance-wide default is used. The never value keeps them without expiry.
• artifacts:when is on_success by default. Use always to upload test reports from failed jobs too.
• cache:key:files builds the key from file contents (at most two files). If none of the files exist, the key is default. Changing the lock file gives a new cache, which is the point.
• cache:policy is pull-push by default. pull only downloads, push only uploads. The caching documentation describes a pattern with one job that fills the cache and pull-only jobs that read it.

needs: DAG pipelines

By default, stages set the order. needs builds a directed acyclic graph (DAG): a job starts as soon as the jobs it lists have finished, even if other jobs in earlier stages are still running. needs: [] starts a job immediately. By default a job fetches artifacts from jobs in previous stages; needs:artifacts and dependencies narrow that to the jobs you name.

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - make build

lint:
  stage: test
  needs: []
  script:
    - make lint
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

unit:
  stage: test
  needs:
    - build
  script:
    - make test

deploy:
  stage: deploy
  needs:
    - unit
    - job: lint
      optional: true
  script:
    - make deploy

If a needed job is missing from the pipeline because of rules, pipeline creation fails with an error such as ‘unit_tests’ job needs ‘compile’ job, but ‘compile’ does not exist in the pipeline. In the example, lint exists only in merge request pipelines, so deploy marks it optional: true. The other choice is to give both jobs matching rules.

Reuse: extends, !reference, include

Three tools remove copy and paste. extends merges hashes from a template job, and arrays are replaced, not merged. !reference pulls one part of another job, such as its script. include loads other YAML files (for example local, project, remote and template).

include:
  - local: 'ci/build.yml'
  - project: 'my-group/ci-templates'
    ref: main
    file: 'templates/deploy.yml'

.setup:
  script:
    - echo "shared setup"

.base:
  image: alpine:3.20
  variables:
    LEVEL: base

job-a:
  extends: .base
  script:
    - !reference [.setup, script]
    - echo "level is $LEVEL"

• When keys overlap, the main .gitlab-ci.yml takes precedence over included files.
• Plain YAML anchors work, but not across files pulled in with include. !reference can point to configuration in included files.
• The documentation advises avoiding more than three levels of extends because of complexity.

Environments, manual gates, variables

The environment keyword names the target a job deploys to. If the environment does not exist when the pipeline runs, it is created. A manual gate is when: manual in a rule.

deploy-production:
  stage: deploy
  script:
    - ./deploy.sh production
  environment:
    name: production
    url: https://example.com
  rules:
    - if: $CI_COMMIT_TAG
      when: manual
      allow_failure: false

A manual job inside rules defaults to allow_failure: false, which is a blocking manual job. The pipeline stops at that stage with status blocked until someone runs it. Outside rules, the default is true, so the pipeline does not wait. Blocked pipelines cannot be merged when Pipelines must succeed is enabled. Combine a blocking manual job with a protected environment and only users in the Allowed to deploy list (or GitLab administrators) can run it.

Variables and secrets

• Protected variables are available only to pipelines on protected branches or protected tags. Use them for production credentials.
• Masked variables show as [MASKED] in job logs. The value must be a single line, at least 8 characters, with no spaces. GitLab states that masking is not a guaranteed way to stop a malicious user from reading a value.
• Store tokens and passwords in the project settings, not in .gitlab-ci.yml. Values set in the UI take precedence over variables in the YAML file.

Troubleshooting

Validate before you push. Check YAML syntax with a linter first. Then use the CI lint tool in Build > Pipeline editor > Validate. It checks syntax including include files, and has an option to simulate pipeline creation. The simulation runs as a push event on the default branch, so it does not reproduce merge request or tag pipelines.

Official documentation

• CI/CD YAML syntax reference
• Specify when jobs run with rules
• workflow keyword
• Merge request pipelines
• Caching
• Job artifacts
• needs keyword
• Use include
• Optimize GitLab CI/CD configuration files
• Job control and manual jobs
• Environments
• CI/CD variables
• Predefined variables
• Validate CI/CD configuration

Keep going