1.3.0 Upgrade Guide

What's New

Visual Validation

A big thanks to @jkeys089 who contributed this after exploring various commercial and open-source tools. Here are the highlights (in his own words):

Some of the issues we encountered using external screenshot comparison services include:

• Difficulty sharing a single set of baseline images across feature branches (e.g. developers working on different features will run into failures until their updates can be included in the set of baseline images)
• Hosted services are limiting for remote developers (e.g. it isn't possible to run screenshot comparisons without a fast, reliable internet connection)
• Hosted services have inherent limitations (e.g. limited number of screenshots with costly overage penalties when using commercial services or single-threaded performance using an OSS solution locally)

The solution:

• Run screenshot comparisons in realtime as we take them (even when running multi-threaded tests)
• Define comparison settings inline with the tests where the screenshots are taken
• Review comparison results and modify screenshot settings directly in the Karate reports
• Check-in baseline screenshots and comparison configs with the tests (e.g. so developers could make updates in feature branches independent of other branches)

Karate now has a compareImage keyword and the corresponding karate.compareImage() JS API.

Refer to ( this video TODO ) for how to use the HTML UI in the Karate report to inspect, configure and update the screenshots.

Graal JS multi-thread issues are solved

This is a big deal, achieved after upgrading Graal to version 22.0.

Advanced users of Karate may have run into some edge cases when trying to pass a JavaScript function to called feature files, especially when callonce and karate.callSingle() are involved and tests are run in parallel. These issues were mostly solved in 1.1.0 and 1.2.0, but a few rare cases were still reported.

This issue is finally resolved along with some code clean-up and we are back to how things were in v0.9.X. You can freely pass JS functions all over the place.

New karate.response and karate.request API

This specifically solves for retrieving a given header while ignoring the case. While Karate already had support for this in simple match statements and via the configure lowerCaseResponseHeaders option, there were advanced use-cases that required more control. You can find more details here. Here is an example:

karate.response.header('content-type').

This also makes mock request routing based on headers much easier, for e.g. karate.request.header('foo') == 'bar'.

New option to write mocks in JavaScript

This is an alternate option for those who want to write more complicated mocks and opens up a lot of possibilities. The "server side" JS API is simple, clean and designed to even serve dynamic HTML.

Refer to this documentation for more: Karate JavaScript Mocks.

Easier way to drive dynamic Scenario Outlines

See the new @setup life-cycle described below.

Breaking Changes

@setup life-cycle

This is an important change that adds a new life-cycle to scenarios. There is a description and discussion here. The updated documentation can be found here.

The highlights are:

• adds a way for data to be set-up before a Scenario starts
• the focus is on returning data, so no "global" state modifications are allowed, which keeps things simple
• this was introduced specifically to make it easier to setup a JSON array (or function) for Dynamic Scenario Outlines
• so you can think of this as a scenario that acts as a "background" for a Scenario Outline (but can also be called from any Scenario)
• the Dynamic Scenario Outline had an inconsistency, which is that the Background was only run once, but with this change, the Background will run before every Scenario whether it is
  • a normal Scenario
  • a row from a fixed set of Examples: in a "normal" Scenario Outline:
  • or a row generated at runtime by a dynamic Scenario Outline: <-- this is the breaking change

Here is a diff of what to expect. In most cases, where you were using a Background to "drive" a dynamic Scenario Outline, the change is to use a Scenario tagged with @setup instead.

Some JS behavior has changed

ℹ️ Ignore this if you have not referred to JS functions within other JS functions.

For details, see: https://github.com/karatelabs/karate/issues/2009#issuecomment-1228632313

Websocket support has changed

ℹ️ Ignore this if you have not used the karate.webSocket() API

This is a breaking change, but the pattern for editing your existing tests is quite straightforward. Here below is a before-and-after:

• the socket.listen(5000) call has to be replaced with * listen 5000 - where 5000 (here just an example) is the timeout in milliseconds
• the magic-variable listenResult holds the result of the captured websocket message
• more details are in the docs: https://github.com/karatelabs/karate/tree/develop#websocket

Heads Up

1.3.0 will be the last release of Karate that allows for usage of Java 8. From 1.4.0 onwards, Karate will have a minimum requirement of Java 11. Please comment here if you have any concerns.