Static Analysis and Dynamic Analysis

By Boguslaw Czwartkowski, Parasoft Professional Services Manager

Static code analysis is a broad term used to describe several different types of analyses:

• Static code analysis (a.k.a. pattern-based static analysis): This type of static analysis looks for code patterns that violate defined coding rules. In addition to ensuring that code meets uniform expectations for regulatory compliance or internal initiatives, it also helps teams prevent defects such as resource leaks, performance and security issues, logical errors, and API misuse.

• Flow-based static analysis: This type of static analysis involves finding and analyzing the various paths that can be taken through the code, both by “control” (i.e. the order in which lines can be executed) and by data (i.e., the sequences in which a variable or similar entity can be created, changed, used and destroyed).  This can expose problems that lead to critical defects such as memory corruptions (buffer overwrites), memory access violations, null pointer dereferences, race conditions or deadlocks. It can also detect security issues by pointing out paths that bypass security-critical code (for example, code that performs authentication or encryption).

• Metrics analysis: This involves measuring and visualizing various aspects of the code. It can help detect existing defects, but more often it warns of potential difficulty in preventing and detecting future defects when code is maintained. This is done by finding complexity and unwieldiness such as overly large components, excessive nesting of loops, too-lengthy series of decisions and convoluted intercomponent dependencies.

Why Static Code Analysis is so Valuable

The thing all these types of static analysis have in common is that they involve scanning (i.e., having a program examine) source code. This is a very fast and easy way to expose critical defects. It achieves 100% coverage and its results are 100% objective. It's almost impossible to argue against doing it continually.

Beyond the Scope of Static Analysis

Static scanning provides information to help predict what may happen when code is integrated and executed. It detects defects based on what the tool considers a defect (typically, this can be customized to suit your preferences and priorities). However, it cannot tell you when the system under test or in production does NOT do what it is expected to do—or does something that nobody expected it would do.

The challenge here is observing the unexpected behavior. For example a transaction may appear to proceed correctly to a user (or tester or test execution tool) whereas a component has actually thrown an unhandled exception and failed to process it correctly. A control system may respond quickly and correctly under test for 3 days, yet be leaking memory and heading for a crash on its 4th day in production. Fixing all defects detected by using a static code analysis tool gives no assurance against other defects that will cause failures like these.

That's why it's important to apply the definition of failure to internal as well as external behavior, even after integration. The internal failure must be detected before it manifests itself externally.

Dynamic Analysis - Runtime Error Detection

At Parasoft, we refer to this activity as runtime error detection, a form of dynamic analysis. Here, distinctions among testing types start to blur. Dynamic application security testing is analytical testing in that the intention is to examine the test item rather than exercise it. It is white box testing in that we examine internal rather than external behavior. Yet the code under test must be executed, and that is done by running the same black box tests used for dynamic testing.

Runtime error detection detects and reports an internal failure at the instant it occurs, so it is easy for the tester to precisely correlate it with test actions for incident reporting. Like good static analysis, it provides full technical details to enable the developer to isolate and fix the underlying defect. Runtime error detection extends the capability of empirical testing at all levels, from unit to acceptance, by making it possible to detect an internal failure that indicates an otherwise-unobservable external failure has occurred or will occur after testing has stopped.

***

Photo Credit: Dexxus