Follow the Path of the Test Pyramid to Achieve Continuous Testing

As software development grows more complex with things like microservices, containers, and APIs, and the demand for faster releases increases, testing as usual can be seen as a clumsy, out-of-place step in the delivery process. In this post, learn how to turn it around.

Achieving continuous testing isn’t like following the yellow brick road… or is it? Although it sounds simple, there are various tests and traps and tricks to getting where you need to go. Organizations that have adopted Agile development and DevOps methodologies know that it’s hard for testing to keep pace with these modern development practices. Tools that don’t enable things like test automation and continuous testing are hindering their efforts.

But like Agile and DevOps, with Continuous Testing there is no one right answer for doing things, and that can make it hard for organizations to understand clearly where to start. Doing some level of test automation from a functional perspective is foundational for continuous testing. Service virtualization is a key enabling technology for continuous testing; however, for you to even be able to take advantage of service virtualization, and really become a true continuous testing practitioner, you have to have some level of test automation in place to start with.

Many organizations will start by simply automating UI testing, but to enable continuous testing, organizations need to focus on creating a scalable test automation practice that follows the ​testing pyramid,​ as defined by Mike Cohn (in his book, Succeeding with Agile) and Agile thought leader Martin Fowler. At the base of the pyramid is a foundation of unit tests, which are isolated and faster to execute. As you move up the pyramid, the assets you’re testing are more integrated, and take longer to perform. The middle of the pyramid is where you do service integration, or API testing, which is the sweet spot for functional test automation, and then end-to-end UI testing is the smallest piece, topping off the pyramid.

Optimize for API Testing to Hit the First Phase of Continuous Testing

If you want to achieve continuous testing, a foundation of unit tests is a given, and API testing becomes the critical layer for validating your use-cases. You’ll also want to optimize your use of end-to-end UI tests at the top of the pyramid. You shouldn’t eliminate UI tests, but you want to make them as efficient and focused as possible, because you have to worry about maintaining those tests, which can be quite time-intensive.

Having done this, organizations get to the first phase of value from continuous testing — earlier-stage identification of regressions — and the best way to find those regressions is at the API level, because they’re quicker to execute, easier to diagnose, and easier to reproduce than their UI level cousins.

Unlocking the Second Phase of Value from Continuous Testing

Once you have your unit and functional tests running, and you’re using service virtualization to run your regression tests more continuously, you can take a look at your non-functional business requirements — e.g. performance and security.

In the case of performance for example, it’s very easy for development teams to introduce incremental performance issues in the application without even realizing it, and it’s critical to catch these issues early in the development process to ensure that the application’s overall performance characteristics aren’t going outside of your defined service level of agreement (SLA).

This is where you can take advantage of your continuous testing practice in order to shift-left performance testing. Start by applying service virtualization to isolate your part of the system and remove the performance bottlenecks in the test infrastructure. Then take your robust set of API tests and reuse them to generate load on your part of the overall system. Finally, integrate the execution of the performance tests into your CI pipeline and execute these tests continuously. This is when you really start getting your second phase of value from continuous testing.

Parasoft Technologies that Make Continuous Testing a Reality

At Parasoft, our tools span unit, functional, UI, and regression testing, in order to help our customers deliver high-quality software that also meet end-user wants and needs. We are working on everything from unit and functional testing of the API and the UI, test data management, safety and security compliance, change management, and more.

Following the test pyramid, you can start at the base and get Parasoft C/C++test, Parasoft dotTEST, and Parasoft Jtest development testing tools, covering such things as static analysis for uncovering deep reliability and security (with support for the OWASP, CWE, and CERT standards), unit testing, coverage, and traceability.

As you travel up the pyramid, you can leverage Parasoft SOAtest, our functional testing solution that focuses on validating applications at the API level. To help organizations move from manual testing to automated API testing, you can use SOAtest’s Smart API Test Generator, which leverages AI and machine learning to build test scenarios from data relationships extracted from recorded application traffic​.

To really deliver on the promise of continuous testing, you can then leverage service virtualization with Parasoft Virtualize, which lets organizations decouple test automation from constraints within the test environment. This enables true continuous testing, making it possible for automated tests to be continuously executed throughout the CI pipeline.