1.3.0 Upgrade Guide

What's New

Visual Validation

A big thanks to @jkeys089 who contributed this after exploring various commercial and open-source tools. This was designed to solve issues encountered using existing / external screenshot comparison services. For example:

• Difficulty sharing a single set of baseline images across feature branches
  • e.g. developers working on different features will run into failures until their updates can be included in the set of baseline images
• Hosted services are limiting for remote developers
  • e.g. it isn't possible to run screenshot comparisons without a fast, reliable internet connection
• Hosted services have inherent limitations
  • e.g. limited number of screenshots with costly overage penalties when using commercial services
  • single-threaded performance using an OSS solution locally

The solution:

• Run screenshot comparisons in realtime as we take them
  • even when running multi-threaded tests
• Define comparison settings inline with the tests where the screenshots are taken
• Review comparison results and modify screenshot settings directly in the Karate reports
• Check-in baseline screenshots and comparison configs with the tests
  • developers can make updates in feature branches independent of other branches

Karate now has a compareImage keyword and the corresponding karate.compareImage() JS API.

Refer to this video for how to use the HTML UI in the Karate report to inspect, configure and update the screenshots.

Graal JS multi-thread issues are solved

This is a big deal, achieved after upgrading Graal to version 22.0.

Advanced users of Karate may have run into some edge cases when trying to pass a JavaScript function to called feature files, especially when callonce and karate.callSingle() are involved and tests are run in parallel. These issues were mostly solved in 1.1.0 and 1.2.0, but a few rare cases were still reported.

This issue is finally resolved along with some code clean-up and we are back to how things were in v0.9.X. You can freely pass JS functions all over the place.

New karate.response and karate.request API

This specifically solves for retrieving a given header while ignoring the case. While Karate already had support for this in simple match statements and via the configure lowerCaseResponseHeaders option, there were advanced use-cases that required more control. You can find more details here. Here is an example:

karate.response.header('content-type').

This also makes mock request routing based on headers much easier, for e.g. karate.request.header('foo') == 'bar'.

New option to write mocks in JavaScript

This is an alternate option for those who want to write more complicated mocks and opens up a lot of possibilities. The "server side" JS API is simple, clean and designed to even serve dynamic HTML.

Refer to this documentation for more: Karate JavaScript Mocks.

contains only deep for match

This is an enhancement to match that makes it possible to assert that a JSON is "deep equal to" another - but with the slight twist that JSON array order is ignored. Come to think of it, we should have had this sooner :| This is expected to be very relevant for teams using GraphQL. Details here: https://github.com/karatelabs/karate/issues/2093

configure abortSuiteOnFailure

Some teams have requested for being able to stop the entire test suite if one test fails. This will save time when the environment has issues and "fail fast" instead of letting the CI job plough on and result in all tests failing. Details here: https://github.com/karatelabs/karate/issues/2090

Easier way to drive dynamic Scenario Outlines

See the new @setup life-cycle described below.

Breaking Changes

@setup life-cycle

This is an important change that adds a new life-cycle to scenarios. There is a description and discussion here. The updated documentation can be found here.

The highlights are:

• adds a way for data to be set-up before a Scenario starts
• the focus is on returning data, so no "global" state modifications are allowed, which keeps things simple
• this was introduced specifically to make it easier to setup a JSON array (or function) for Dynamic Scenario Outlines
• so you can think of this as a scenario that acts as a "background" for a Scenario Outline (but can also be called from any Scenario)
• the Dynamic Scenario Outline had an inconsistency, which is that the Background was only run once, but with this change, the Background will run before every Scenario whether it is
  • a normal Scenario
  • a row from a fixed set of Examples: in a "normal" Scenario Outline:
  • or a row generated at runtime by a dynamic Scenario Outline: <-- this is the breaking change

Here is a diff of what to expect. In most cases, where you were using a Background to "drive" a dynamic Scenario Outline, the change is to use a Scenario tagged with @setup instead.

Some JS behavior has changed

ℹ️ Ignore this if you have not referred to JS functions within other JS functions.

For details, see: https://github.com/karatelabs/karate/issues/2009#issuecomment-1228632313

Websocket support has changed

ℹ️ Ignore this if you have not used the karate.webSocket() API

This is a breaking change, but the pattern for editing your existing tests is quite straightforward. Here below is a before-and-after:

• the socket.listen(5000) call has to be replaced with * listen 5000 - where 5000 (here just an example) is the timeout in milliseconds
• the magic-variable listenResult holds the result of the captured websocket message
• more details are in the docs: https://github.com/karatelabs/karate/tree/develop#websocket

Heads Up

1.3.0 will be the last release of Karate that allows for usage of Java 8. From 1.4.0 onwards, Karate will have a minimum requirement of Java 11. Please comment here if you have any concerns.