Want to see Parasoft in action? Sign up for our Monthly Demos! See Demos & Events >>
X
Want to see Parasoft in action? Sign up for our Monthly Demos! See Demos & Events >>
Reading Time: 7 minutesContinuous testing provides a framework to shift testing earlier in the life cycle. It is possible to streamline testing and make it less of an inhibitor in Agile and continuous processes. What does it take? The right application of automation and focus on the highest risk areas of the application with the use of containers for secure software development.
Containers play an important role in shift-left testing. And in the success of continuous testing and DevSecOps in general. They’re also becoming helpful in development as containerized development and testing environments provide benefits like those in deployment. Here are a few:
There’s an important role for containers to play in modern DevSecOps development as containers can help shift-left security processes in software development.
The Accelerating DevSecOps With Containers and Continuous Testing whitepaper details leveraging containers for secure software development to achieve security at speed. This post looks at the following key areas discussed in the whitepaper to better integrate containers into the development and testing phases of software development:
The goal is to achieve better security while minimizing the impact on everyday workflows.
By taking advantage of the consistency and reusability that containers provide, continuous security testing works best when each developer and tester on the team works from the same configuration. Moving tools and associated configurations into containers ensures consistent use and any changes propagate immediately. A smarter DevSecOps workflow means that security is “baked in” from the start.
Security requirements add additional workload for developers. How you manage makes the difference between an on-time, secure product, and a late, insecure one. A critical requirement is to integrate security into the existing development process —making it less of a burden and part of the quality and the overall workflow.
The workflow starts with a secure coding policy. The Architect or Lead creates a configuration (possibly based on coding guidelines such as CERT, CWE, OWASP driven by standards like UL-2900 or PCI DSS) for the rest of the team to leverage directly within their IDE. This gives the developer the ability to check the code locally on their machine before committing to source control. They can catch and fix security violations where and when it’s cheaper and easier to do so.
The same configuration is replicated through container images, providing consistency, then leveraged by analysis executed as part of the build process. This comprehensive analysis goes beyond the scope of the developer’s locally modified code and provides a safety net to gate the delivery pipeline to make sure that insecure code does not get promoted to later stages.
Lastly, the results of the analysis are sent back to the developer’s IDE via the centralized reporting and analytics dashboard. This is where you can track progress, make course corrections, and generate audit reports in real time.
Managers and security leads can assess projects based on security standards such as CWE Top 25, OWASP Top 10, or SEI CERT in central dashboards. These dashboards can show trending information and answer questions like:
Being able to answer these and other questions — and act — transforms the development team from DevOps with some security to DevSecOps.
Despite the desire to accelerate security and quality, Agile and continuous processes hit barriers in early adoption and in day-to-day operations. The culprit in slowing down Agile processes is often testing. The problem arises because teams don’t understand exactly what to test during each iteration. They either try to test everything, don’t test nearly enough, or simply guess. All three outcomes are unacceptable, particularly from a security standpoint. Smart test execution further helps teams focus their testing efforts, truly enabling continuous testing.
The test pyramid defines where it’s best to invest time and effort into a project. In the ideal pyramid, you invest your valuable time and effort into a comprehensive suite of unit tests at the foundation of the pyramid, which is backed up by API and service tests. A smaller number of system and GUI-based testing is at the top of the pyramid. These are tests that require human intervention, such as UX testing.
This pyramid is often inverted into an ice cream cone shape. Teams spend too much time and effort on brittle and complex system-level GUI tests that require implementing and integrating full functionality. The result is testing that teams cannot execute continuously during the earlier stages of the SDLC.
This is especially problematic with security. It’s costly and risky to leave security testing to such a late stage of development. The key to achieving successful continuous testing is to melt the ice cream cone. Focus on creating automated unit and API tests for functionality and security that can be executed continuously as developers are implementing the new functionality.
The best approach to turn the test pyramid right-side-up — which aligns nicely with the best practices needed for DevSecOps — is to adopt these testing best practices:
API-level testing provides the ability to test system functionality with more reusability and less impact from changes at the UI level. This means a lot of data is generated by the automated process. To support decision making, this data must be aggregated into a centralized view of quality. Graphical dashboards provide a top-level view of testing progress. You can create additional work items based on review of the data, identified defects, and other issues.
A way to focus testing is missing from the best practices listed above. It begs an answer to the question, “What to test?” Focusing testing on exactly what developers need to review after each code change is critical to accelerating testing, enabling continuous testing, and expediting the DevSecOps pipeline.
To accelerate testing in an Agile or continuous pipeline, teams need smart test execution on a per-build basis. It reduces the required sets of tests that teams need to execute to address the risk that each new iteration introduced. The analytics that test impact analysis provides are key to focusing testing on only what you absolutely need to test rather than the shotgun approach.
Only through smart, can data-based decision making enable continuous testing. Focusing the development team on the minimum set of tests to ensure proper coverage at each iteration is the key to bring the agility back to Agile development methods.
Smart test execution uses test impact analysis to trace the execution of manual tests against applications and associated, captured code coverage information with those tests. Automated testing uses similar technology. This analysis figures out which manual tests need execution to access changed functionality delivered with every new build. Thus, smart test execution is critical at the developer and tester levels in their local IDEs. It enables them to focus the testing and remove the guesswork and extra “just in case” work.
To achieve an efficient DevSecOps pipeline, developers need access to automation that removes the tedious aspects of their day-to-day work. Successful security — and quality and safety — development requires quick feedback from unit testing, API testing, UI testing, and SAST results. Plus, fine-tuning security controls based on the most recent findings by the security team.
The benefit of containers for development teams extends to the developer’s desktop. Containers provide a consistent and reusable toolchain (including testing/SAST configuration) across developer, build, integration, and deployment environments. They eliminate the need to update local workstations each time configurations are updated. In container-based development, only the container image needs to be updated. The entire team benefits.
Team-based tool deployments can use containers to host repositories, different toolchains (especially important for embedded software development), SAST, and test tools. An illustrative example of a possible containerized development environment is shown below.
Figure 4: Containers provide an efficient way for developers to have a consistent development and testing environment.
Where developers really start to accelerate secure development is with local “workstation” adoption of smart test execution. Reinforcing good security and engineering practices (to perform security analysis and test execution locally and often) between code modifications and before committing any changes to the source control system. This process significantly reduces the feedback loop on the security and quality of modified code. It generally speeds up the “code, test, submit” workflow.
Development teams can use a continuous testing workflow to help focus testing efforts and make sure they’re performing only necessary tests. In addition, constant monitoring of code security with SAST tools is critical to ensure that teams adhere to coding standards and that tricky vulnerabilities aren’t eluding security tests.
Quick feedback and analysis of SAST and test results reduces the friction in continuous testing. The consistency provided by container-based development environments means results are the same across the team. It reduces the need to “debug” different development environments when results don’t match. Bringing the power of smart test execution to the developer’s desktop truly accelerates secure software development.
Containers are a well-established execution and deployment approach. They also support rapid deployment and portability across different host environments with support for versioning and centralized control. CI/CD are also important for secure development in a DevSecOps pipeline since it’s possible to provide a reproducible application environment with built-in security controls.
Continuous testing provides a framework to shift testing earlier in the life cycle. With the right application of automation and focus on the highest risk areas of the application, it’s possible to streamline testing to be less of an inhibitor in Agile and continuous processes. The use of containers for secure software development extends benefits of versioning and reusability to the developer desktop by providing a consistent toolchain, testing/SAST configuration, development and (local) deployment, and test environment.
Arthur has been involved in software security and test automation at Parasoft for over 25 years, helping research new methods and techniques (including 5 patents) while helping clients improve their software practices.