Automation fails frequently in companies due to a variety of reasons, including poor team communication, lack of skills, flaky tests, and inadequate understanding of test coverage. Even when things are going well, the automated tests sometimes grow to a size where the test suites take too long to execute for the run to be viable. James Farrier is a test automation architect who will show you ways to leverage machine learning to address these challenges. You'll learn how to determine which tests are valuable to run after each commit or build in order to cut down the suite run time, how to automatically close and open defects based on test run results, and how to separate a test into different test runs to keep track of tests in different states. Finally, he will show you how to create a results dashboard that allows for team collaboration and a better understanding of test coverage so that testing can be further streamlined.

Some developers and testers think that APIs, whether internally developed or not, should just work. Unfortunately, without a significant amount of focus on testing API services, one little flaw can spell disaster for your application—especially security vulnerabilities where hackers will target your application. Elise Carmichael discusses the risks of not fully testing APIs, how to be sure that all services are tested fully, and the types of negative testing you need to do on APIs. She shows you a method to organize your tests using Postman, a tool for REST web services testing. During this session, Elise and the participants will write some Postman API tests with variables and then execute the tests with Newman, a command line application for running Postman tests that you can include in continuous integration scripts.

A difficult challenge organizations face when adopting continuous integration/continuous delivery (CI/CD) is ensuring that every build has been evaluated for performance. Although most organizations do performance testing, how they do it and when they do it define their readiness for deploying in fast-paced environments. Anjeneya Dubey’s team faced similar challenges and found ways to embed performance engineering as part of their CD pipeline. To achieve this objective, they made changes to processes, technologies, and culture. They added non-functional requirements to “normal” requirements and built a performance engineering platform for each API. This platform automates the performance tests, provides a centralized database for metrics, and automatically analyzes pass/fail on builds, based on dynamic thresholds.

Most software organizations have a test automation initiative—some just beginning and others humming right along. Typically, these initiatives focus on traditional automation—using a software package to automate test cases or user stories. However, if you use only the traditional approaches to automation, you will miss opportunities to exploit the complete powers of automation. Paul Grizzaffi shares real-world examples where non-traditional automation or “automation assist” approaches, including high volume automated testing, have provided high business value testing in his company. He highlights instances where traditional methods could not have provided the same level of testing. Paul shares thoughts to keep in mind when implementing automation assists and demonstrates examples of helpers he’s created from tools not traditionally used for test automation.