Testing 1-2-3 With Parasoft’s Mark Lambert: Change-Based Testing and Merged Correlated Code Coverage

Last time we featured our inaugural session with voke's Theresa Lanowitz, where we discussed  Continuous Testing, Service Virtualization & the New Role of Testers.

In this edition of Testing 1-2-3, we chat with Mark Lambert, VP of Products at Parasoft about change-based testing, merged correlated code coverage, and Agile development.

Agile and Change-Based Testing

Testing 1-2-3:  Here at StarWest, everyone is talking about change. Many people are adopting new ways of developing and testing software in response to demands for faster delivery and better visibility into the risks associated with the software that's being released.  How are you seeing this play out in the field?

Mark: The #1 thing that I'm seeing lately is that people want to know the impact of the frequent changes being made to their applications. You might have tests that cover the entire application, but—especially with Continuous Delivery—you can't run all of those tests to understand the impact of each change. With Agile, everything revolves around 2 or 3 week sprints, and you're really focused on the functionality that's being introduced in each user story. Of course you validate each story and make sure it works, but do you really know what else you might have impacted? Exploratory testing might uncover something, but there's usually no methodical way to know whether those changes had dangerous side effects.

Testing 1-2-3:  So in other words, the challenge today is getting feedback at change time so that you know whether or not you can move forward with confidence?

Mark: Yes, it's all about the scope of the impact.

Testing 1-2-3:  Interesting. How can people take the first steps towards transitioning to this new paradigm?

Mark: I think the first step is getting more information. If you lack some of the underlying data around your existing testing practices, how can you expect them to evolve, become agile, and be leveraged in a way where they help you mitigate the risk of each release candidate?

Merged, Correlated Code Coverage

Testing 1-2-3:  Can you give us an example of that data?

Mark: One example is understanding what parts of the code are actually touched by specific test cases. At Parasoft, one of things we're focused on is helping organizations get that test traceability—the ability to understand what code a particular test exercises. Most people are familiar with measuring coverage at the unit level, but you also want to get a granular level of code coverage for your manual and automated tests as well.

Testing 1-2-3: But those are all separate practices performed at separate times throughout different phases of the SDLC. You have someone doing unit testing when the code changes (or maybe even before the code changes, if they're taking  a TDD approach), you have somebody performing component testing, and you have a different group of people coming in to perform scenario tests predicated on the requirement or the user story itself. Given all this, how do you determine what level of coverage was really achieved?

Mark: Although all these practices used to be very siloed in different parts of the SDLC, now they're really all condensed into a particular sprint. You need to be able to aggregate the test data and coverage information from all of these different techniques being performed simultaneously against a particular build. The correlation is vital…you bring all this data together, correlate it against the specific build going through the verification process, and then you merge the coverage across those different techniques. At Parasoft, we call this merged correlated coverage.

Release = MC2 (Merged Correlated Coverage)

Want to learn about how merging and correlating code coverage from a variety of testing techniques can help your organization understand and mitigate the risks associated with a given release? Read our whitepaper, Comprehensive Code Coverage: Aggregate Coverage Across Testing Practices.

You'll learn about how application coverage helps teams meet their release objectives, including:

• How application coverage helps mitigate the risk associated with accelerated delivery
• How to collect application coverage
• How to merge and correlate code coverage from a variety of testing techniques
• How application coverage can help safety-critical organizations achieve the traceability necessary to meet compliance objectives