Checks and Assertions

In this guide you will learn how to use checks and assertions within your test cases to make your tests easier to debug and reason about.

The only thing you need to follow along this guide is a StormForger account (sign up here).

As always, if you have any questions, feel free to get in contact any time!

Motivation

Test cases can get quite extensive and complex. When building up your cases or executing them, its good to see if certain assumptions are (still) correct.

Checks and assertions operate in the context of one client session. One example would be that within the user journey the login request has to be successful before you can continue with further steps. This can be done with checks and assertions.

You will see a summary of used checks and assertions in test run reports.

Checks

A check is a concept to see if a certain condition is true or not. Checks in StormForger work pretty much like session.if(). They have

  • a name
  • a variable to check (e.g. session.getVar("csrfToken"))
  • a comparator (=, !=, >, >=, < or <=) and
  • an value to compare against.

If the check fails - read: the condition is not met - we will count a "KO" under its name and a "OK" otherwise. OKs and KOs will be summed up across all sessions.

Here is an example test case:

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

session.check("login", session.getVar("accessToken"), "!=", "");

The above example could generate the following "Checks" part in a test run report:

Reported checks

You can take a look at a demo report where we have counted some OKs and KOs.

Assertions

Assertions behave very much like checks with one major difference: After counting a KO assertions will terminate the current session. If the condition within an assertion passes, an OK is counted. Think of session.assert() as session.check() combined with session.abort().

We can now use session.assert() to improve the example from above:

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

session.assert("login", session.getVar("accessToken"), "!=", "");

// will only execute if "accessToken" is present!
session.get("/personal-profile");
// ...

If the scenario (like ours here) cannot be continued without a successful login, session.assert() can be used to make sure that the scenario is aborted for this user. This helps to minimize noise caused by lots of consequential errors arising because of a failed authentication step.

In addition to counting a KO, you will also get a summary of reasons (based on the assertions name) why a session was aborted in the test report:

Reported checks and session aborts

Shorthands for HTTP Errors

If you are interested to see if the request was successful in terms of HTTP semantics, you can achieve something similar using the abort_on_error request option which is a bit more concise in this case:

session.get("/", {
  abort_on_error: true,
});

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  tag: "login",
  abort_on_error: true,
});

Using abort_on_error we will terminate the current session if the response code of the request is HTTP 4xx or 5xx. The abort reason (name) will be named after the requests tag - login in our case. If no tag is provided, http_error will be used.

Reported abort_on_error

There is also a way to switch this on automatically for all requests. This will basically add a abort_on_error: true for all subsequent requests:

session.defaults.setAbortOnError(true);

session.post("/login", {
  payload: {
    user: "road-runner",
    pass: "meep meep",
  },
  extraction: {
    jsonpath: {
      "accessToken": "authorization.token",
    }
  },
  tag: "login",
});

See also our reference on Session Checks.

Get started icon

Get Started

New to the StormForger?
With these guides you’ll be up and running in no time!

FAQ icon

FAQ

Already took a look at our FAQs?

Support icon

Support

Are you stuck? Talk to us! We're humans.

We are using cookies to give you the best online experience. If you continue to use this site, you agree to our use of cookies. By declining we will disable all but strictly required cookies. Please see our privacy policy for more details.


Accept Decline