Writing API Tests with Karate E-Book

Benjamin Bischoff

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 368

Veröffentlichungsjahr: 2023

Beschreibung

Software in recent years is moving away from centralized systems and monoliths to smaller, scalable components that communicate with each other through APIs. Testing these communication interfaces is becoming increasingly important to ensure the security, performance, and extensibility of the software. A powerful tool to achieve safe and robust applications is Karate, an easy-to-use, and powerful software testing framework.
In this book, you’ll work with different modules of karate to get tailored solutions for modern test challenges. You’ll be exploring interface testing, UI testing as well as performance testing.
By the end of this book, you’ll be able to use the Karate framework in your software development lifecycle to make your APIs and applications robust and trustworthy.

Leseprobe

Writing API Tests with Karate

Enhance your API testing for improved security and performance

Benjamin Bischoff

BIRMINGHAM—MUMBAI

Writing API Tests with Karate

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Gebin George

Publishing Product Manager: Kunal Sawant

Senior Editor: Rounak Kulkarni

Technical Editor: Pradeep Sahu

Copy Editor: Safis Editing

Project Coordinator: Deeksha Thakkar

Proofreader: Safis Editing

Indexer: Manju Arasan

Production Designer: Shyam Sundar Korumilli

Developer Relations Marketing Executives: Sonia Chauhan and Rayyan Khan

Business Development Executive: Kriti Sharma

Production reference: 0130423

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83763-826-0

www.packtpub.com

I dedicate this book to my wife, Anne, and my daughter, Frida, who gave me so much love, time, and support to finish this project without going crazy. I love you! Thanks a million!

– Benjamin Bischoff

Foreword

It gives me great pleasure to write this foreword for the first-ever book dedicated to Karate. 

My journey with Karate began with a simple desire to help teams struggling with test automation. I have always been driven by the belief that there had to be a simpler, better, and more maintainable way to automate tests. As the creator of Karate, I am indeed thrilled to see how far it has come, and to witness the impact it has had on teams and enterprises all around the world. I am grateful to the community that has embraced and contributed to Karate’s growth, making it a powerful tool for organizations.

What started out as a small side-project focused on API testing is now an open-source company, Karate Labs. Since the first version of Karate was released six years ago, we are trusted by 1000’s of organisations and are aware we have so much more to do as we scale Karate Labs as a responsible open-source company. 

Hand-on-heart, I have always wondered whether I should write a book on Karate. So, when I came to know that Benjamin is writing a book on Karate, I felt a sense of relief. There’s nothing better than having an accomplished thought leader write a book on Karate. I deeply respect Benjamin’s work in the test-automation and open-source domain and had the pleasure of hosting him on one of our customer webinars. You can count on him to give you an informed, balanced perspective - and with great attention to detail.

Thank you, Benjamin - for your dedication and expertise in crafting this guide to Karate. I am confident that this book will be an invaluable resource for both beginners and seasoned professionals, enabling them to harness the full potential of Karate and take their test automation efforts to new heights.

Sincerely,

Peter Thomas

Co-founder of Karate Labs and creator of the Karate test-automation framework

Contributors

About the author

Benjamin Bischoff decided in 2017 to make test automation his main career after being a full-time developer for 15 years. He works as a test automation engineer in Trivago’s product foundation backend QA team, where his focus lies on backend, API, and database automation, as well as build and test pipelines. Prior to that, he took care of frontend automation and the development and maintenance of Trivago’s in-house end-to-end test framework. Benjamin is the author and maintainer of two open source projects for Cucumber BDD parallel test execution and reporting. He is also a conference speaker and writes about testing and automation topics on his website, https://www.softwaretester.blog.

I would like to thank the people who were close to me and gave me feedback. My brother, my parents, and the entire Trivago QA team, especially Saurav Sharma for his technical contribution and Karan Tohan for convincing me about Karate. I also thank Peter Thomas and Kapil Bakshi, for all their work and dedication. Thank you, Packt team, for all the help!

About the reviewer

Saurav Sharma is a seasoned software quality assurance professional with over 12 years of experience in the field. Currently, he works as a QA lead for Trivago at their headquarters in Düsseldorf, Germany, where he collaborates with an excellent team of QA professionals, focusing on backend services. Originally, he is from Katras, a small town in the Dhanbad district of Jharkhand, India, and holds a BTech degree in information technology. He enjoys technical and people leadership and has worked in various capacities with web and mobile applications for companies including Oracle, Rocket Software, Allscripts, and Tech Mahindra. He strongly believes that test automation and CI are vital for ensuring speed and quality in software development.

Table of Contents

Preface

Part 1: Karate Basics

1

Introducing Karate’s Core Concepts

Technical requirements

What makes Karate stand out?

Discovering Karate’s strong points

Core features

BDD versus Karate

What is BDD?

Glue code

The Karate way

Supported data types in Karate

JSON

GraphQL

XML

YAML

CSV

Other text-based formats

Binary formats

The JavaScript engine

Java interoperability

Summary

2

Setting up Your Karate Project

Technical requirements

Installing Java

Setting the JAVA_HOME environment variable

Getting to know Karate standalone

Preparing the IDE

Setting up Visual Studio Code

Setting up IntelliJ IDEA

Importing the example project

Setting up Karate projects with Maven

What is Maven?

Installing Maven

Adding Maven to the PATH variable

Setting up a Karate project with the Maven archetype

The roles of the different Karate project files

Summary

3

Writing Basic Karate Tests

Technical requirements

Exploring the API under test

The JSONPlaceholder API

Creating a new Karate project

Adding a new feature file

Configuring test runs

Adding a scenario

Calling endpoints and setting parameters

Setting a URL

Specifying the HTTP method

Separating the base URL from the path

Setting query parameters

Matching status codes and responses

Matching the status code and type

Making the test fail

Using assertions and matchers on the response

Exploring the response variable

Handling nested JSON elements

Asserting the number of elements with the length property

Using matchers

Making requests with payloads

Using variables and data tables

Using variables

Using data tables

Using a set

Summary

4

Running Karate Tests

Technical requirements

Running and debugging Karate tests through the IDE

Running via CodeLens and the Karate CLI

Debugging via CodeLens and Karate standalone

Using the debug server

Running tests with Maven

Understanding Karate runners

Running tests against different environments

Using a custom property in a feature file

Using Karate’s environment property

Setting up karate-config.js

Running specific tests

Running specific runner class methods

Running specific scenarios

Filtering tests by tags

Summary

5

Reporting and Logging

Technical requirements

Configuring log options

Understanding log levels

Changing the log level

Suppressing print

Using Karate’s built-in reports

The different report pages

Preserving old reports

Deciding what to report

Configuring third-party report libraries

Setting the appropriate Karate options

Using Cluecumber as a Maven plugin

Technical requirements

Using Cluecumber from a Karate runner

Generating a Surefire report

Surefire XML

Using JUnit reports

Summary

Part 2: Advanced Karate Functionalities

6

More Advanced Karate Features

Technical requirements

Working with headers and cookies

More about headers

Setting headers

More about cookies

Using different configuration and run options

Using the karate object for configuration and execution

Request retries

Using advanced tags

Defining and using expressions with def

Defining inline methods

Using embedded expressions with JSON

Using embedded expressions with XML

Working with external files

Understanding JSONPath and XPath

Testing GraphQL

Understanding GraphQL requests

Exploring the mock API

Using GraphQL in Karate

Collecting data into a new array

Summary

7

Customizing and Optimizing Karate Tests

Technical requirements

Using Karate hooks

Implementing a new hook class

Overriding handler methods

Registering a hook class in the runner

Running a test using the hook class

Defining hooks inside feature files

Defining and calling Java functions

Understanding the basics

Working with databases

Using Karate as a mock server

Authoring a mock scenario

Firing up the mock server from within a test

Using a standalone mock server

Making your tests more concise

Reducing code by calling other feature files

Data-driven tests with scenario outline

Avoiding code duplication with background scenarios

Summary

8

Karate in Docker and CI/CD pipelines

Technical requirements

Triggering Karate tests from shell scripts

Creating a batch script for Windows

Creating a Bash script

Running Karate tests in a Docker container

Understanding Docker

Installing Docker

Starting and verifying a Docker installation

Downloading a Docker image

Running a Maven command in a Maven Docker container

Running Karate tests inside a Maven Docker container

Running our shell script inside Docker

Customizing our tests

Passing database credentials as system properties

Using the system properties in the database connection class

Passing parameters from Docker

Environment variables

Integrating Karate tests into GitHub workflows

Understanding GitHub workflows

Managing secrets

Adding a GitHub workflow to a repository

Setting up the Docker-based GitHub workflow

Using Karate in a GitHub workflow without Docker

Summary

9

Karate UI for Browser Testing

Technical requirements

What Karate UI is for

Related testing capabilities

Writing a basic Chrome scenario

Opening a website in Chrome

Finding and interacting with elements

Understanding locators

Knowing the different kinds of locators

Determining element locators

Interacting with elements

Using interactions in a test scenario

Making web element assertions

Making web element list assertions

Debugging UI tests

Taking screenshots

Highlighting elements

Exporting PDFs

Using the debugger

Using karate.stop to pause execution

Redirecting HTTP calls

Investigating API requests

Adding a mock response

Configuring the mock

Using the mock in a test

Summary

10

Performance Testing with Karate Gatling

Technical requirements

Creating the test scenario

What is Gatling?

Setting up Karate Gatling

Setting up Scala in VS Code

Creating Maven profiles

Running Karate tests as performance tests

Creating a simulation

Running the simulation

Using tags

Testing different simulations

Gatling assertions

Checking out Gatling reports

Summary

Index

Other Books You May Enjoy

Preface

The Karate framework is a relatively new software testing solution that aims to bridge the gap between exploratory testing and code-heavy automation. It has a strong focus on being easy to learn but still providing complete functionality for API testing and beyond.

This book first explains the core concepts and ideas of the framework, how to set up new test projects, and how to write effective API tests. It further covers the various ways to run these tests in different settings and environments and explains various reporting and logging options, including completely custom ones. It also covers advanced Karate features, such as extending the functionality of the framework with custom JavaScript and Java code and even using it in Docker and continuous integration/continuous delivery (CI/CD) pipelines.

In addition, the book covers two of Karate’s lesser-known capabilities: Karate UI for browser testing and performance testing with Karate Gatling. This comprehensive guide is a useful resource for anyone looking to learn about the Karate framework and implement it in their software testing practice.

Who this book is for

If you are a QA engineer or developer who is already familiar with APIs and wants to cover them through automated testing, this book is for you. Even if you have done other forms of testing and want to know more about API testing, this book can help you understand the core ideas and approaches of these tests. And even if you already use other frameworks and techniques for API testing, you will learn here what sets Karate apart from its competitors. To that end, you’ll find a lot of information in this book to help you decide whether Karate can be a good replacement for your current toolset.

What this book covers

Chapter 1, Introducing Karate’s Core Concepts, introduces the Karate testing framework, which provides an overview of its strengths, key concepts, and special features.

Chapter 2, Setting Up Your Karate Project, provides a step-by-step guide for setting up a new Karate test project, including integrated development environment (IDE) preparation, dependency management with Maven, and basic configuration using the Karate configuration file.

Chapter 3, Writing Basic Karate Tests, introduces writing the first API tests using Karate, which examine return codes and responses while ensuring code efficiency through Karate and Gherkin mechanisms.

Chapter 4, Running Karate Tests, explores various methods for triggering and running Karate tests, including considerations for running tests in CI/CD pipelines, selecting test cases, and executing tests in parallel to optimize runtime.

Chapter 5, Reporting and Logging, gives an overview of Karate’s built-in reporting and logging capabilities for effective troubleshooting, plus a guide for integrating third-party reporting solutions.

Chapter 6, More Advanced Karate Features, explores concepts and techniques for creating more complex test cases, including setting and checking headers, cookies, and authentication tokens, testing GraphQL APIs, and switching between different test environments.

Chapter 7, Customizing and Optimizing Karate Tests, shows how to create custom functionality in Karate through JavaScript functions and Java interoperability, as well as how to write custom Karate hooks to react to Karate events.

Chapter 8, Karate in Docker and CI/CD Pipelines, integrates Karate tests into CI/CD pipelines to establish a fully automated test setup using the example of GitHub workflows.

Chapter 9, Karate UI for Browser Testing, introduces Karate UI, a special module for browser-based test automation, and explores how this alternative approach to user interface (UI) testing fits into the Karate ecosystem.

Chapter 10, Performance Testing with Karate Gatling, explores the integration between Karate and the popular Gatling framework for load and performance testing, which reuses existing Karate scenarios.

To get the most out of this book

Basic Java and JavaScript knowledge, as well as a basic understanding of the testing methodology, will help you to find your way around faster. However, in this book, I also try to cover the required aspects in a way that is understandable even without this knowledge.

Software covered in the book

Operating system requirements

Java SDK

Windows, macOS, or Linux

Maven

Windows, macOS, or Linux

Karate Standalone

Windows, macOS, or Linux

Google Chrome

Windows, macOS, or Linux

IntelliJ IDEA

Windows, macOS, or Linux

Visual Studio Code

Windows, macOS, or Linux

Postman

Windows, macOS, or Linux

MySQL

Windows, macOS, or Linux

phpMyAdmin

Windows, macOS, or Linux

Git

Windows, macOS, or Linux

Docker

Windows, macOS, or Linux

You will not need any commercial software or tools to follow along. I deliberately wanted to use free and open source software so everyone can use it without spending additional money.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Writing-API-Tests-with-Karate. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/IIe3X.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Gherkin files have the .feature extension and always start with a Feature: line followed by a description.”

A block of code is set as follows:

Package blog.softwaretester.gherkin; import io.cucumber.java.en.Given; import io.cucumber.java.en.Then; import io.cucumber.java.en.When;

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

public class StepDefinitions {    @Given("I am on the Web shop homepage")     public void goToHomepage() {        System.out.println("Go to homepage");    }

Any command-line input or output is written as follows:

java -cp C:\Users\bbischoff\Desktop\karate-1.2.1.RC1\karate.jar com.intuit.karate.Main

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “To trigger one of the saved requests, click Send.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Writing API Tests with Karate, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere? Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application. 

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781837638260

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Karate Basics

In this part, we will introduce the ideas and concepts of Karate and examine what sets it apart from other frameworks. We will set up the system and learn how to create new test projects using Maven. After that, we will start writing our first tests and explore different ways to run them. Finally, we will take a closer look at the reporting and logging options built into Karate and go through how you can integrate your own reports.

This section contains the following chapters:

Chapter 1, Introducing Karate’s Core ConceptsChapter 2, Setting Up Your Karate ProjectChapter 3, Writing Basic Karate TestsChapter 4, Running Karate TestsChapter 5, Reporting and Logging

1

Introducing Karate’s Core Concepts

The software testing world is constantly changing. Technologies that were recently state of the art are replaced or turn out to be slow sellers. New frameworks are created all the time to try to solve common testing problems. There are a lot of people who have entrenched opinions about how tests should look, and sometimes this attitude can even block progress. Testing is often still divided into exploratory testing (also called manual testing, which I personally dislike) and code-heavy automation solutions (also known as test automation), which may require a software expert for development and maintenance but might not be feasible for more hands-on exploratory testing.

In between these extremes, there is plenty of room for innovation. There has been quite some progress here during the last few years, and these new technological solutions are in high demand.

The Karate framework is a rather new contender in the software testing field that has set out to solve many of the problems raised, especially the steep learning curve of other solutions.

We are going to cover the following topics:

What makes Karate stand out?Behavior-driven development (BDD) versus KarateSupported data types in KarateThe JavaScript engineJava interoperability

Technical requirements

All code examples for this chapter can be found on GitHub at https://github.com/PacktPublishing/Writing-API-Tests-with-Karate.

Demo code

It is not necessary to set up the programming environment at this point. All code that is shown here is just to demonstrate Karate’s inner workings and core functionality. In the next chapter, we will deal with this thoroughly!

What makes Karate stand out?

At its core, Karate is a test automation framework that is written in Java but does not necessarily require programming skills when used for basic software testing. It is heavily based on the Gherkin syntax, made famous by the Cucumber BDD framework. In fact, when it was first introduced to the software testing world, it was designed as an extension of Cucumber but quickly evolved into something of its own. However, some of Cucumber’s core characteristics are still evident and can be put to good use. For example, the very similar file structure, syntax, and compatibility with the standard JSON reporting format.

The first version was released on February 08, 2017, by Peter Thomas while he was working as a test engineer at Intuit India (https://www.intuit.com/in), a large software company in the fintech industry. His motivation was to come up with an easier-to-use and easier-to-understand API testing solution than the in-house framework his team was using at this time.

In Peter’s own words,

Karate strives to reduce the entry barrier to writing a test and more importantly - reduces the friction to maintain a test, because of how readable tests become.

Thomas, P. (2017). Karate: Web-Services Testing Made Simple, https://medium.com/intuit-engineering/karate-web-services-testing-made-simple-366e8eb5adc0

Intuit suggested making his work open source in order to test how this new approach would be received by the community.

At the time of writing this book, the Karate framework sports more than 6.6k stars on GitHub (https://github.com/karatelabs/karate) and even received a GitHub grant for open source software in India.

Peter has apparently struck a nerve with the testing community!

In 2021, after this successful open source launch and adaption period, Peter decided to found Karate Labs (https://www.karatelabs.io), together with Kapil Bakshi, in order to work full-time on the further development and promotion of the framework and to offer some new paid products and services around Karate.

No paywall

The paid services by Karate Labs include subscription-based plugins for IntelliJ IDEA and Visual Studio Code with multiple tiers. In this book, however, we will focus exclusively on the freely available core framework and its various open source components and tools.

Interest in the Karate framework has not waned since then, as can be seen by the plethora of questions on Stack Overflow and its GitHub issues page. Since a lot of information is spread among many different online articles, social media channels, blog posts, and podcasts, I decided to write a book about it to provide a structured, comprehensive, and complete introduction.

In the upcoming sections, we will discover how Karate works in more technical and conceptual detail and what distinguishes this solution from other popular testing frameworks that may be more specialized but are also much harder to learn and use.

Also, we will further explore the similarities and differences between the classic Gherkin syntax and underlying concepts and Karate’s different approach in the next section. We will find out why certain decisions were made by the framework creator and why they were smart moves.

Discovering Karate’s strong points

Karate’s strengths lie in various areas of test automation, which we will discuss in this section. Later on in the book, we will shine a light on these different areas with a strong focus on the framework’s core purpose – testing APIs effectively.

Since there are multiple modules of Karate with different goals, let’s quickly walk through them to see how extensive the framework and its components have become over the years.

Testing APIs

Providing a basis for API tests that is easy to read and follow is the main capacity of Karate – it supports the REST, SOAP, and GraphQL APIs and can be extended to handle other types of APIs and data formats that are not as common. All other framework modules that cover different kinds of testing were gradually added and implemented with the same basic mechanisms and concepts. By focusing on its straightforward domain-specific Gherkin-based language, Karate can map a lot of API test functionality without having to write elaborate code for communicating with web services or handling various payloads and data exchange formats.

Additionally, Karate includes many powerful assertions to verify response data, and its structure, especially JSON and XML, are handled exceptionally well. This makes it a perfect fit for a lot of modern APIs that use these formats.

We will explore all of this in Chapter 3, Writing Karate Tests.

Fun fact

Even though Karate is often associated with JSON, XML support was implemented first in its initial prototype before JSON handling was added!

Test doubles

Testing microservice infrastructures or web services calling other services or databases can get very complex and challenging. Typically, the approach is to spin up all components that make up a full system in order to test it thoroughly. This makes it hard to run tests for a single component because each test needs to go through multiple layers. This means that other dependent services need to be very predictable and reliable in such a setup so you can obtain meaningful and non-flaky test results.

Test doubles, a term coined by agile software development consultant Gerard Meszaros (http://xunitpatterns.com/gerardmeszaros.html), solve this problem. It groups together all forms of fake objects (or fake APIs) that can be used instead of real objects in tests. They behave as if you were making real requests and data transfers but deliver the same predictable responses every time. This allows you to perform testing on a small part of the system without having to directly use all real dependencies and services. Thus, the tests can run faster and more securely because you are not affected by network problems, bad connections, database inconsistencies, and other failures.

Karate can define such doubles easily using nothing but Gherkin, as we will see in Chapter 6, More Advanced Karate Features!

Performance tests

Karate’s main strength in the performance testing field is that it can reuse existing scenarios. The same test cases that are used as acceptance tests can seamlessly act as Gatling performance tests. This means that it is possible to simulate multiple clients or connections that interact with your APIs in order to check their resilience and how they hold up under heavy loads. By integrating the popular Gatling framework, Karate gains a huge advantage here because it frees us from using completely separated solutions, third-party frameworks, and libraries that would require additional research, learning, and onboarding.

Also, due to its ability to reuse already implemented test scenarios, we do not need to spend time and resources duplicating them in another programming language just for the sake of performance tests.

In Chapter 10, Performance Testing with Karate Gatling, we will look deeper at this important testing topic.

Browser automation

The automation of websites and browser-based user interfaces has become much more important in recent years. More and more user-facing software is available as web applications, which can be very complex in design. Also, a lot of functionality and logic can be wired into the frontend, making it necessary to thoroughly test this area as well.

Karate UI is special here because it combines the Chrome DevTools Protocol (CDP) and the WebDriver protocol, so it can accomplish pretty much everything that other popular tools such as Selenium, Cypress, and Playwrightcan do.

As we will see in Chapter 9, Karate UI for Browser Testing, this module was one of the last ones added to the Karate framework to cover this testing field as well. Among other things, it provides full debugging capabilities and even allows freely mixing API and UI automation to handle even more complex use cases.

Desktop automation

Karate also supports automating desktop applications through its integration with the Robot framework. This is an entirely different use case than the other ones that are discussed in this book, but maybe this will contribute to your decision to give Karate a try.

If you want to know more about this part of Karate, check the documentation at https://github.com/karatelabs/karate/tree/master/karate-robot.

Now that we have gotten to know Karate’s core uses, let’s find out about its main features next.

Core features

Since many useful and often needed functions of other specialized test frameworks are already implemented in the core framework, you will reach useful results fast and without a steep learning curve. This makes Karate ideal for quickly implementing a large number of test cases and working off low-hanging fruits (things that take only a little time while having a great benefit).

Karate’s big advantage is the consistent syntax across test types. This makes switching between different test projects and contexts easier and faster, as there is no need for extensive training. Also, it keeps the test suites very clean, easy to understand, and maintainable when written well.

If, however, you require expanded functionality, you are free to add it as either Java or JavaScript code, depending on which language features can solve your problem better and your respective skills with each of them.

Let’s look at some of its most useful aspects!

Parallel test execution

Often, extensive test suites contain a lot of different scenarios that differ greatly in individual runtime. Executing these tests in sequence, meaning one after the other, can lead to an unnecessarily long feedback time. Many other test frameworks have parallelization built in, at best, as an afterthought or require additional tools for this. Also, it can be complex to execute and evaluate (for example, because test reporting cannot deal with parallel scenarios correctly). Karate has this approach on board as a basic feature for all different test types.

As we will later see in Chapter 4, Running Karate Tests, we will even be able to visualize and track how tests are run on different threads, making this a powerful option for more efficient test runs.

Data-driven testing

In Chapter 3, Writing Basic Karate Tests, we will discover that the Karate framework even supports data-driven tests. That means that one or more test scenarios can be reused with multiple sets of data instead of writing separate ones for each individual use case. This reduces code duplication and errors caused by having to change multiple code locations across a lot of feature files if requirements change.

The unique aspect of Karate in this regard is that these records of data can even be defined in JavaScript Object Notation (JSON) or comma-separated values (CSV) formats. Also, they are not required to be static data as in a Cucumber-based framework but can be generated dynamically.

Auto-generated test reports and logs

Karate has its own built-in reporting library that generates informative HTML-based test reports that show what features and scenarios were run along with requests, responses, and additional information such as tags, screenshots, and even a timeline of parallel runs. These help greatly in analyzing test runs and can act as a form of documentation as well. In case you need to display custom information, Karate even includes a full templating engine that can be programmed to display any additional data you may need.

Additionally, all Karate actions, up to every single request and response, are logged. Even with the most beautiful test reports, it is often simple text logs that provide the most information about errors that have occurred.

We will encounter both variants of debugging, including custom configuration and extension, in Chapter 5, Reporting and Logging.

First-class support of common data formats

Karate, through its Java and JavaScript roots, can handle pretty much any data format found on the web today. This does not only mean processing them but also making assertions, performing schema validations, and efficient fuzzing matching of results. Later in this chapter, we’ll take a closer look at the formats that are natively supported. Additionally, we will explore this topic further in Chapter 3, Writing Basic Karate Tests, and Chapter 4, More Advanced Karate Features.

Beginner friendliness

One of the best characteristics of the Karate framework is that it allows users to start using it as a solution where no coding is necessary and gradually realize more complex test cases by writing Java or JavaScript code on top of it. The Karate domain-specific language (DSL) will be discussed in more detail a little later in this chapter.

When using Karate, it is worth mentioning that it is usually just one single dependency you need that covers most functionality (for example, API testing, UI testing, and performance tests), making this rather simple to set up.

Additionally, there is even a standalone executable version of Karate that can do everything that the Java library can do but without requiring experience with setting up a Java project. This is useful for quickly trying out Karate without extensive preparation. For a production environment, I would nonetheless recommend the Maven setup that we will see in Chapter 2, Setting Up Your KarateProject, and Chapter 4, Running Karate Tests.

Built-in support of different environments

If you have a mature test suite, the goal is typically to run it within a continuous delivery pipeline on different development environments, for example staging, user acceptance, and live. Often, these environments have different URLs, endpoints, access rules, and configurations. So, it is very convenient if the test framework offers a simple way to keep these configurations inside the test repository and switch easily between them.

Karate offers this as a native functionality as we will see in Chapter 6, More Advanced Karate Features.

Read-eval-print loop

A read-eval-print-loop (REPL) is a programming environment that reads user inputs, then processes and executes them as code snippets and prints the results. Karate has a command-line REPL, which allows test code to be inserted or replaced at runtime, making it easier and faster to develop tests. This can save time that would otherwise be spent modifying, adapting, and executing a complete test scenario repeatedly. The REPL is a main component of Karate’s Visual Studio Code plugin and can also be used for UI tests!

One framework for all

All previously mentioned features are anything but self-evident, and it is an accomplishment not to be undersold that they come together here in one integrated software testing solution!

In this section, we looked at Karate’s main use cases and features that make it stand out from other frameworks. Next, we will turn to an aspect of the framework that is controversial to some testers – the difference between the BDD approach and how Karate takes this idea and changes it.

BDD versus Karate

Now that we know about the core ideas and features of the Karate framework, let’s look at how it achieves writing meaningful test scenarios without extensive coding.

What is BDD?

BDD comes from the world of agile software development. It focuses on bringing together the different roles in a software project (for example developers, testers, and project owners) and talking about requirements. These requirements should then be put into clear scenarios so that, later, all roles can take these scenarios as a basis for their work.

A core part of BDD is a ubiquitous and simple DSL that can express desired behavior in natural language and is understood by every team member. Its purpose is to form a basis for the development of new features but also be a clear guideline for testing and acceptance. In the following sections, we will look at one of the most well-known and widely used DSLs – Gherkin.

The Gherkin language

If you have been in touch with the Cucumber Open BDD testing tool (https://cucumber.io/), you will have stumbled across the Gherkin language. Gherkin is a very concise and straightforward way to turn acceptance criteria into understandable and well-structured test scenarios. At its core, it is a plain-text format to describe how a system under test should behave. Cucumber can parse this format and link it to custom test code, so it provides a readable wrapper around the technical implementation of tests. This way, other development and QA team members, as well as people with non-technical roles, can read these specifications and know exactly what is expected from the system.

The following is an example of the contents of a typical Gherkin feature file (named webshop.feature):

Feature: Web shop tests     @smoketest     # This is an example comment     Scenario: Product search works as expected         Given I am on the Web shop homepage         And I am logged in         When I search for 'Packt'         Then I get a list of items containing 'Packt'

Even though this feature file is taken from a website test (commonly called UI test), Gherkin can be used to describe any behavior of any system under test – potentially even cases that have nothing to do with software at all.

Let’s take a look at the different parts of the preceding file.

Features

Gherkin files have the .feature extension and always start with a Feature: line followed by a description. Each feature file collects test scenarios that belong to a defined generic business case indicated by the given description. In our example, the feature name Web shop tests shows that this is a collection of test cases for our imaginary online store.

Scenarios

A feature can include one or more scenarios that are associated with it. Each scenario is a test case related to a specific business case, user journey, or behavior.

Scenarios are made up of different keywords and parts that we will look at in the following sections.

Tags

Within feature files that contain multiple scenarios, you can use one or more custom tags to further group them into test suites (in our case, the tag @smoketest could mean that the following scenario belongs to a set of small, basic tests that are run after a deployment to quickly verify that our application is usable). In the course of this book, you will see how these can be used to pre-select the tests you want to run at a specific point in your software development lifecycle.

Steps

Scenarios are made up of different steps that are executed from top to bottom. Each one describes either the state of the system under test, certain actions, or assertions. Their generic nature makes them excellent for reuse across multiple test cases.

Keywords can be substituted

From the processing of the scenarios by a Gherkin-based framework, it does not matter at all which keywords are used in which order. They are used solely for comprehensibility and semantic correctness. In any case, you should get into the habit of following these conventions.

Next, let’s look at the different types of Gherkin keywords you can use.

The Given keyword

This keyword indicates the initial state and prerequisites of an application, its user, or data. In our example, Given I am on the Web shop homepage tells us where the users of the application start their journey in this test case. It is deliberately kept simple to increase comprehensibility and does not contain any technical details.

The When keyword

Any interactions with or changes to the system under test start with a When keyword. When I search for 'Packt' describes clearly what the users do and what their intention is but not how the webshop deals with this on a technical level. Later, we will see that Karate bends this core BDD rule and why it does it this way.

The Then keyword

The final part of most test scenarios is one or more assertions to verify a certain outcome or state of the system under test. This is expressed by Then as in Then I get a list of items containing 'Packt'. Again, this clearly states what outcome is expected from the system but not how exactly this is solved by the application.

The And keyword

The And keyword can connect multiple Given, When, or Then lines depending on if there is more than one prerequisite, action, or assertion. It is just “syntactic sugar” to make a scenario more readable.

In our example it connects two Given statements:

The user is on the homepage and……the user is logged in

The But keyword

Like And, the But keyword can be used to connect steps. It is not as common as its counterpart but can express certain conditions better. Take the following, for example:

Given I am on the Web shop homepage

And I am not logged in

That can also be formulated as follows:

Given I am on the Web shop homepage

But I am not logged in

This would emphasize even more that the focus of this test is on an unauthorized user.

Catch-all steps (*)

Gherkin has one more step keyword that all different step types (Given, When, Then, And, and But) can be substituted with. A * step is mostly used when it does not really fit the natural flow of what should be tested but instead contains utility functions or setup code, or helps with debugging test scenarios.

An example is Karate’s print function, which logs a string on the command line for simple tracing of values:

* print "The value of my variable is", myVariable

Here, it does not directly play a role in any part of the tested behavior. Instead, it can be considered as a utility step that still has to follow the chronological sequence of events but could easily be separated from them without affecting the overall test functionality. In Karate, an advantage of this is that these bullet points can be suppressed in the generated test reports, as we will later discover.

Comments

Comments can be used to make certain steps clearer, especially when they contain information that might be hard to understand for newer team members or colleagues that are not as familiar with the tested business domain.

Also, they can be used in case a test is temporarily deactivated to explain further why this was done or what needs to happen before it can be active again.

Comments can be placed anywhere in a Gherkin file. They start with a # symbol (like the # This is an example comment line in the example) and are ignored by Cucumber and Karate alike.

Additional Gherkin syntax

In this section, we only looked at the basic Gherkin keywords that will be necessary in the first steps toward writing test cases.

The Gherkin specification has some more advanced language constructs such as the following:

Data tables: For specifying multiple sets of data for a stepExample tables: For running scenarios with different sets of dataBackground scenarios: That define common steps in multiple scenariosScenario outlines: For combining similar scenarios into oneDocstrings: For more complex data definition in a step

All of these will play a role in later chapters when we look at code reuse, keeping tests free from code duplication, and the overall structuring and fragmentation of test suites

Writing good BDD scenarios

When doing BDD, test scenarios should typically have the following characteristics to keep them clear, understandable, and concise across your test suites:

Scenarios should be able to run independently