Automate the tool qualification process for safety critical software

Automate the tool qualification process to reduce the time and effort required, so you can focus on developing high-quality software.

Safety-critical software development standards, as such as DO178B/C (aerospace), ISO 26262 (automotive), EN-50128 (railway), and IEC 61508 (functional safety), require that manufacturers prove that the tools they are using to develop their software provide correct and predictable results. The process of providing such evidence is known as Tool Qualification, and, while it's a necessary process, Tool Qualification is often a tedious and time consuming activity that many organizations fail to plan for.

The end deliverable is proof in the form of documentation, but there’s more to the qualification process than just delivering a big pile of static documentation. In this post, I’ll show you how to use Parasoft's new Qualification Kits for C/C++test, which include a convenient tool wizard that brings automation into the picture and reduces the time and effort required for tool qualification.

First: make sure the tool you're using is certified

Tool Qualification needs to start with tool selection, ensuring you are using a development tool that is certified by an organization an such as TÜV SÜD. This will significantly reduce the effort when it comes to Tool Qualification. Parasoft C/C++test is certified by TÜV SÜD for functional safety according to IEC 61508 and ISO 26262 standards for both host-based and embedded target applications – paving the way for a streamlined qualification of static analysis, unit testing, and coverage requirements for the safety-critical standards.

Second: reduce the manual effort of tool qualification

Traditionally, tool qualification has meant significant amounts of manual labor, testing, and documenting development tools to satisfy a certification audit. But this documentation-heavy process requires manual interpretation and completion, and as such, is time-consuming and prone to human error.

We're solving this problem by bringing automation to this process with our Qualification Kits. Increasing productivity and reducing human error through automation has always been at the core of what we do at Parasoft, so we’ve applied this same mission to the Tool Qualification process for Parasoft C/C++test. Our Qualification Kits walk the user through an intuitive workflow to dramatically reduce the amount of effort required.

Benefits of using the Qualification Kits include:

• Automatically reducing the scope of qualification to only the parts of the tool in use
• Automating tests required for qualification as much as possible
• Handling any manual tests as eloquently as possible, and integrating results alongside automated tests
• Automatically generating audit-ready documentation that reports on exactly what’s being qualified, not more (or less!)

Qualify only what you use

Why make developers sift through qualification material for DO-178B/C when they are doing an automotive project requiring ISO 26262? What happens if you’re only using parts of our tool suite? There should be no need to do any extra work for qualifying capabilities not used in your development products. Reducing the scope of testing, reporting, and documentation is a key way to reduce the qualification workload.

The example below shows a user selecting the only use-case of C/C++test being used (static analysis) to check compliance to the MISRA C 2012 standard, as part of ISO 26262 qualification. The tool then selects only the parts of the qualification suite needed for this function.

Users can deselect options to further reduce the scope as required for their project. Upon selection, only tests and documentation is used and provided from this point forward.

Leverage test automation and analytics

If there’s one advantage to qualifying test automation tools... it's that the tools can be used to automate their own testing! The tool qualification requires that tests be run and results verified as outlined in the qualification kits we provide. Automating this as much as possible is key to making it as painless as possible. Even manual tests, which are inevitable for any development tool, are handled as efficiently as possible – step by instructions are provided and results are entered and stored as part of the qualification record.

Parasoft C/C++test collects and stores all test results from each build, and tests run as they do for any type of project. These results are brought into the test status wizard in the Parasoft Qualification Kits to provide a comprehensive overview of the results:

Centralizing, aggregating ,and automating the qualification process greatly reduces manual tracking of compliance progress.

What about known defects?

Every development tool has known bugs and any vendor that doesn’t document them isn’t doing your compliance effort any favors. But there’s more to dealing with known defects than just documenting them. Tool qualification requires that you provide proof that these defects are not affecting the results used in the project. For each known defect, you must provide a mitigation for each one and document it to the satisfaction of the certifying auditor.

It’s incumbent on the tool vendor to automate the handling of known defects as much as possible. After all, we are asking customers to deal with our bugs as part of their workload! The Parasoft C/C++test qualification kits includes a wizard to automate the recording of mitigation for known defects as shown below:

Automate the Documentation You Need

The end result of tool qualification is documentation, and lots of it. Every test executed with results, every known defect with mitigation, manual test results, and exceptions are all recorded and reported. Qualification kits from other vendors can be just documentation alone, and without automation, documenting compliance is very tedious.

Instead, using the Qualification Kits for C/C++test, the critical documents are generated for you as part of the workflow:

• Tool Classification Report (determines the qualification needed, and presents the maximum safety level classification for C/C++test based on the use cases selected by the user)
• Tool Qualification Plan (describes how C/C++test is going to be qualified for use in a safety relevant development project)
• Tool Qualification Report (demonstrates that C/C++test has been qualified according to the tool qualification plan)
• Tool Safety Manual (describes how C/C++test should be used safely, i.e. compliant to safety standards like ISO 26262, IEC 61508, etc. in safety critical projects)

In each of these documents, only the documentation required for the tool features in use is generated, since the scope of the qualification was narrowed down at the beginning of the project. Automation and narrowing the scope of qualification greatly reduces the documentation burden.

Conclusion

Tool qualification for safety-critical software projects in automotive, aerospace, railway, and functional safety industries is a burdensome process. Reducing the scope of qualification and automating as much of the process as possible is key to reducing the workload and tedium of qualification. The Parasoft Qualification Kits for C/C++test make this as painless and efficient as possible, to greatly reduce the burden on our customers, so they can get back to delivering high-quality products to their customers.