Frequently Asked Questions


What is Apdex?

Apdex (Application Performance Index) is an index that summarizes the performance of a system under test.

What is median?

Median is a statistical value, describing the measurement where 50% of all measured values lie below and 50% above. A median response time of 100 ms means that 50% of all requests have been completed within 100 ms and 50% took longer then 100 ms.

What is percentile?

A percentile (or a centile) is a measure used in statistics indicating the value below which a given percentage of observations in a group of observations fall. –

Example: Your test showed that your 95th percentile is 42 ms and your 99th percentile is 150 ms. This means that 95% of your requests were completed within 42 ms (and 5% took longer), and 99% of your requests took 150 ms or less (and only 1% of all requests took longer).

A word on Mean and Standard Deviation

Only looking at mean values for performance measurements is dangerous because you have no indication of how your actual response times are spread around the mean value.

In statistics, the standard deviation […] is a measure that is used to quantify the amount of variation or dispersion of a set of data values. A standard deviation close to 0 indicates that the data points tend to be very close to the mean (also called the expected value) of the set, while a high standard deviation indicates that the data points are spread out over a wider range of values. –

Standard Deviation gives an indication how close the measured response times are to the mean. This gives you some idea how far spread your response times are.

We highly recommend to not use mean as your primary performance indicator. You should take a look at median, 95th and 99th percentile instead.

Making Requests

Which HTTP Headers does StormForger always send?

StormForger by default always sends three headers: User-Agent, X-Stormforger-User and X-Request-Id. See the request tracing section for more details and how to customize these.

Which HTTP Headers can not be overwritten?

You cannot set headers beginning with X-StormForger-.

Why is the number of requests not exactly as defined?

There are several reasons for this.

Most likely the test duration is over before a client has made all its requests. It also appears if a client waits to long for a response and thus can not make the following requests.

Also take a look at Why did my test end early?

Are there any details to the requests defined in a test case?

Yes there are. On the TestCase detail page all defined sessions are listed in the sessions box below the test case summary. If you click on a session a more detailed view will expand. This view contains all defined requests as curl requests which you can and should validate by copying and pasting them in the shell of your choice.

Why are the defined requests not arriving at once in my system?

Although you define a rate in terms of new clients per second in the test case definition StormForger uses a poisson distribution to launch the defined clients.

This is due to the goal of creating a more realistic behavior.

For more details check out the arrival phases reference.

Can I specify which session should use which target?

No you can't. For each session StormForger will use all specified targets in the setTargets definition in a round robin approach. If you need to define different targets in specific requests you have to hardcode an absolute URL in the request like so:



Where will the traffic/requests come from?

We currently only utilize resources from the Amazon AWS cloud. If you need to know what IP ranges will be used, consult AWS's documents on AWS IP Address Ranges. Note that each test-run is executed from a single region which is selectable by the user.

How can I check that traffic is coming from StormForger?

If you want to authenticate StormForger when accessing your testing system, we recommend one of the following:

  1. Use basic auth (or client certificates)
  2. Set a custom defined header
  3. Filter on the standard headers StormForger sends

Note that we do not recommend restricting access to your test system only via the AWS IP ranges, as these IP ranges are quite larger and shared with all customers of AWS.

Why did my test end early?

In general, your test will end if it reaches the defined duration (sum of all phases). But it might end before the defined duration.

Your test will end early (before the configured duration) if:

  1. no more clients should be launched (max_clients has been reached in the last arrival phase) and
  2. all previously launched clients have completed their session

A common reason for irritation comes from max_clients value in your arrival phase. If you launch 10 clients per second, but only allow 300 max_clients your test might end within ~30 seconds (if your session is short).

Let's say you have a very short session (only one request, taking ~1sec). The following test will run for ~5 minutes and 30 seconds:


  // Phase 1 will run for 5 minutes, launching 10 clients per second. It
  // will launch ~3000 clients.
    duration: 5 * 60,
    rate: 10,

  // Phase 2 will launch up to 300 clients (max_clients) with 10 clients per second.
  // After 30 seconds all 300 clients have been launched. If they complete
  // their requests, the test will end.
    duration: 5 * 60,
    rate: 10,
    max_clients: 300


definition.session("very-quick", function(session) {

Why do I see more requests in my logs compared to your report?

We only analyze completed requests currently. So if your test ends before all requests have been completed you might see slightly more requests hitting your target compared to our reporting.

What is Preflight Mode?

Preflight mode means that you will get a fully functional test, but only very limited resources will be allocated to your test. Measurements and reports have to be taken with a grain of salt.

In preflight mode we limit the maximum amount of clients to 300 in your test case.

If you need more load you can send a request via the Request more power button.

Icon Support Are you stuck? Or do you have any feedback? Get in touch with us – we are happy to help you.
Icon Schedule a demo Schedule a personal, customized demo. We'll show you around and introduce you to StormForger.
Icon Talk to a human To build and run reliable applications is complex – we know. Schedule a call and we’ll figure things out.

We are using cookies to give you the best online experience. If you continue to use this site, you agree to our use of cookies. By declining we will disable all but strictly required cookies. Please see our privacy policy for more details.

Accept Decline