How Does Your Unit Testing Process Stack Up?

New Unit Testing Maturity Model

Unit testing is a cornerstone of successful Development Testing strategies and should be adopted by any organization seeking to reduce risks and costs over the application development lifecycle.

As a primary quality gate, unit testing:

• Prevents the introduction of defects that would consume valuable resources if detected later in the development process.
• Reduces risks associated with faulty software, which may result in costly litigation, brand erosion, or even loss of life.
• Verifies the functionality of the application and serve as artifacts for compliance traceability.

Unit Testing processes can range from very simple ad-hoc or reactive efforts to highly-optimized efforts where policies are regularly updated as part of a root-cause analysis effort to prevent defects from being discovered in QA.

To learn more about the benefits of unit testing, please visit: https://www.parasoft.com/solutions/unit-testing/

Ad-Hoc Unit Testing

With ad-hoc unit testing efforts, developers independently choose to create and run unit tests while developing functionality, but tests are not saved or maintained. Instead, they are isolated on independent machines. Ad-hoc unit testing characteristics include:

• Test reports are non-existent or unverifiable.
• Tests are not maintained in an SCM.
• No scheduled automation.
• Reintroduction of defects is accepted as “normal” or “unavoidable.”

Any pockets of maturity at this point are based on the experience and initiative of individuals. There is no centralization of assets; it’s every man for himself.

Tests and test artifacts are typically created as one-off solutions and may or may not be stored on a local machine. Tests are created without consideration of the business or use case.

Signs that it’s time to advance from this level include:

• The application is not adequately tested or integrated due to stated lack of time.
• Business is lost or hard to win due to users’ perception of application instability.
• Returns, uninstalls, and/or complaints tend to interrupt or control development activities

Reactive Unit Testing

The next level beyond ad-hoc unit testing, is reactive, where the team needs to adopt more unit testing, often motivated by external factors such as standards compliance or a major quality problem. At this level, the team and management have committed to unit testing, but implementation is inconsistent across the organization. This level is characterized by:

• Unit tests are maintained in a central repository with the product code.
• Unit tests are run manually during a “smoke test” or “pre-release testing” phase.
• Developers use unit testing to find defects, rather than to assure quality.

At the reactive level, the value of unit testing is recognized, but inconsistent definitions of measurement diminish the value. Since adoption is incomplete, data isn’t shared across teams and it’s difficult to gauge completion or level of quality.

Proactive Unit Testing

Moving beyond reactive testing, organizations realize the need to standardize the use of unit testing across the organization. A common unit testing policy is clearly documented, and teams recognize unit testing as a part of the development process. Developers are standardized on a testing platform to regularly create and extend unit tests during iterations.

This maturity level is characterized by:

• On-target testing is common for embedded systems.
• Scheduled automation of test runs and reports provide traceability.
• The organization uses unit testing to reduce regressions.
• A strategy for unit testing legacy code is adopted.

At this level, organizations start to see real benefits from an organization-wide unit testing policy, usually in terms of a tangible decrease in serious defects. Increased visibility and traceability enables management to make better business decisions. Unit testing is institutionalized as part of the process and is expected development behavior.

Managed Unit Testing

At the managed level of unit testing maturity, organizations start to use a data driven approach to decision making. A metrics-driven policy increases visibility as a centrally-defined process manages unit testing activities. The development team now leverages the testing platform to distribute tasks directly to desktops based on coverage requirements, risk of failure, and other policy-defined metrics.

This maturity level is characterized by:

• Fewer tests fail with each change due to pre-commit desktop runs.
• Developers quickly respond to unit test failures as defined by a policy within a continuous integration workflow.
• Function and object virtualization are used to extend coverage and test error conditions.
• Regressions occur less frequently.

Test-driven development (TDD) becomes a viable option for driving code quality as the value of unit testing increases. Change-based testing becomes a reality because the cost of change is known in advance.

In addition to the increase in quality and security of their software, organizations can start to use the data collected during development to drive better decisions. The organization seeks to leverage the merged and correlated data to perform advanced analytics that identify application hotspots.

Optimized Unit Testing

At the optimized level of unit testing, there is an organizational focus. Policies are regularly updated as part of a root-cause analysis effort to prevent defects from being discovered in QA.

Unit tests are a verification mechanism to verify that policy and process are in sync. Test results are linkable and bi-directionally traceable to all data associated with software and device development. Traceability extends beyond the traditional borders of the SDLC.

Unit test policy is seamlessly integrated into a controlled quality, security, performance, and reliability framework, orchestrated from a centralized interface and inclusive of both development and non-development systems. True Business Intelligence is achieved.

Developers, testers, or managers kick off a test run based on any combination of technical and business requirements. The system automatically appropriates the needed environments, VMs, and tests, then provides results as part of a customizable business intelligence layer.

Unit Testing Maturity Model

This is just a brief introduction to the levels of unit testing maturity. Most organizations today fall somewhere on spectrum of this maturity model.

Parasoft, the leader in software test automation, has developed a unit testing maturity model that provides a detailed look at the 5 different levels of unit testing: Ad-hoc, Reactive, Proactive, Managed, and Optimized.

If you want to assess where your organization currently stands and see what’s involved in moving forward, download the complete Unit Testing Maturity Model.