Using StormForger with GitLab

This guide shows an example GitLab CI/CD workflow utilising StormForger to perform automated load tests after each deployment. We also setup a scheduled load tests (e.g. once a week) against your application.

A validation run is also performed against the staging environment for every push to ensure the load test definition is up to date.

GitLab CI/CD Job Execution

For this guide we assume you have general knowledge of GitLab CI/CD and how it works. You also need the permissions to configure variables in your repository or organisation.

The code is available on GitHub at stormforger/example-gitlab-ci. Our example service is written in Go, but you don't need to know Go, as we will only discuss the StormForger related steps.

Note that this is just an example and your actual development workflow may differ. Please take this as an inspiration how to use StormForger with GitLab CI/CD.


Please follow the Getting Started with the Forge CLI guide to create an API Token and configure it as the STORMFORGER_JWT variable in your repository.

Follow the steps described in GitLab's Creating variables in the UI guide. Since this is a secret, we recommend ensuring it is masked and protected. It should look like this when you are done:

GitLab CI/CD Variable Settings Screenshot

Note: You can also configure secrets on the group level. If you have multiple repositories using StormForger, this might be easier to manage.

Inside the workflow this secret can now be referenced as ${STORMFORGER_JWT}, but you don't need to reference it directly. Our CLI will pick it up automatically.


Our workflow file .gitlab-ci.yml consists of five stages: build, staging:deploy, staging:test, production:deploy and production:test. Everything except the production:* stages are executed on every commit.

Our goal is to run the load test in validation mode against the staging target in the staging:test job and against the production environment in the production:test job. For both environments we follow the same steps:

  1. Manage Data Sources
  2. Build the Test Run and Launch It

There is also a ping step in the build stage to check if the forge cli can communicate correctly with the API.

Let's checkout the ping job first, then go through each step one by one.

Ping with the StormForger CLI

As a first step, we check that the forge CLI works and successfully talks with the StormForger API:

    name: stormforger/cli:latest
    entrypoint: [""]

  stage: build
  extends: .forge
  - forge ping # check that we can reach the API and our token is correct

Here we define a template job .forge that we use in the sf:ping job. Since we will reuse the docker image definition, this avoids having to copy the same image definition multiple times. The stormforger/cli:latest is the docker image that is released as part of our StormForger CLI. You can also copy this image to your private registry, but make sure to update it regularly. The actual work is in the forge ping command: This performs an authorized ping against the StormForger API and verifies that the token is valid and usable.

Manage Data Sources

Data Sources allow a Test Run to pick random data out of a predefined pool, e.g. a product out of all the available inventory. If you don't use data sources, you can skip this step.

In our workflow we use the script ./scripts/ to generate a CSV file for our test, but this can be easily changed or extended to download the latest inventory data from a database.

./scripts/ "${TARGET_ENV}"
forge datasource push ${SF_ORG} ./*.csv --name-prefix-path="${SF_DS_PREFIX}/${TARGET_ENV}/" --auto-field-names

We prefix all uploaded CSV files with our repository name and the target environment to make them easily distinguishable in the data source management of

This step is the same for both the staging and production environment, except for the value of the TARGET_ENV variable.

Build the Test Run and Launch It

Heads Up! Testing in production can be dangerous. Make sure the configured arrival rate/load in your Test Case repo is low enough so you don't trigger any incidents. The goal here is not to run stress tests regularly but to ensure we do not experience any major performance regression with nominal load.

As the last step we launch the Test Run. Here we pass in the environment via --define ENV=\"name\":

forge test-case launch "${TESTCASE}" --test-case-file="loadtest/loadtest.mjs" \
  --define ENV=\"${TARGET_ENV}\" \
  --title="${TITLE}" \
  --notes="${NOTES}" \
  --label="git-ref=${CI_COMMIT_REF_NAME}" \
  --label="gitlab-project=${CI_PROJECT_URL}" \
  --label="gitlab-commit=${CI_COMMIT_SHA}" \
  --label="gitlab-job=${CI_JOB_STAGE} ${CI_JOB_NAME}" \
  --label="giblab-job-url=${CI_JOB_URL}" \
  --label="gitlab-actor=${GITLAB_USER_LOGIN}" \

This allows us to modify the target system urls or just specify global variables in the loadtest.mjs file:

var config;
if (ENV == "production") {
  config = {
    dsPrefix: "example-github-actions/production/",
    host: "",
  }; // production config
} else if (ENV == "staging") {
  config = { ... }; // staging config
} else {
  throw new Exception("unknown environment");

NOTE: To learn more about defining variables or loading modules checkout forge testcase build --help.

The Combined GitLab Job

Combined into a job, the scripts and steps look like this in our .gitlab-ci.yml.

  stage: staging:test
  extends: .forge
  needs: ["sf:ping"]
    - ./scripts/ "${TARGET_ENV}"
    - forge datasource push ${SF_ORG} ./*.csv --name-prefix-path="${SF_DS_PREFIX}/${TARGET_ENV}/" --auto-field-names
    - |
      forge test-case launch "${TESTCASE}" --test-case-file="loadtest/loadtest.mjs" \
        --define ENV=\"${TARGET_ENV}\" \
        --title="${TITLE}" --notes="${NOTES}" \
        --label="git-ref=${CI_COMMIT_REF_NAME}" \
        --label="gitlab-project=${CI_PROJECT_URL}" \
        --label="gitlab-commit=${CI_COMMIT_SHA}" \
        --label="gitlab-job=${CI_JOB_STAGE} ${CI_JOB_NAME}" \
        --label="giblab-job-url=${CI_JOB_URL}" \
        --label="gitlab-actor=${GITLAB_USER_LOGIN}" \

    LAUNCH_ARGS: "--validate"
    NOTES: |
      Head Commit Message:
    TESTCASE: "${SF_ORG}/example-gitlab-ci-${TARGET_ENV_STAGING}"

In this step we attach a lot of information to the Test Launch. The --notes, --title and --label flags provide metadata to enrich the Test Run report.

With LAUNCH_ARGS: "--validate" we are launching the test run only in validation mode for our staging environment.

For the production environment we are instead passing in LAUNCH_ARGS: "--nfr-check-file=./loadtest/loadtest.nfr.yaml" which performs the Non-Functional Requirement checks after the test run has finished.

Scheduled Test-Run Execution

Finally, we want to run the load test once a week automatically via GitLabs pipeline schedules.

Since we don't want to run through all phases (build, test, deploy) nor redeploy to production for this, we use the except statement to disable all jobs. The sf:loadtest_production job gets no except: statement, as this is the job we want to actually execute by the scheduled execution.

For the deploy_staging job, this looks like this:

  stage: staging:deploy
  except: ["schedules"]

Screenshot showing the create pipeline schedule form

You can hit the "Play" button after saving the scheduled pipeline to perform a test execution:

Screenshot showing a scheduled pipeline running the load test

This pipeline will now execute every Sunday night!


To summarise, we used GitLab CI/CD to automatically upload data sources and launch a Test Run for every environment in our development cycle. A weekly job verifies that the load test continuously works and no other factors introduce regressions. By using NFR checks, we automatically verify that our non-functional requirements are fulfilled.

Icon Support Are you stuck? Or do you have any feedback? Get in touch with us – we are happy to help you.
Icon Schedule a demo Schedule a personal, customized demo. We'll show you around and introduce you to StormForger.
Icon Talk to a human To build and run reliable applications is complex – we know. Schedule a call and we’ll figure things out.