Want to see Parasoft in action? Sign up for our Monthly Demos! See Demos & Events >>
X
Want to see Parasoft in action? Sign up for our Monthly Demos! See Demos & Events >>
Reading Time: 4 minutesThe scale of IoT is large — probably larger than most embedded manufactures have previously coped with. Moreover, individual devices are just part of a larger whole that consists of many varieties and types of devices. In my last post, I discussed how IoT’s promise is in the services it provides and the conglomeration of many “things” into something of value to the enterprise. It is important to deal with this scale in both (1) device (“things”) requirements, and (2) the value it adds to the whole automation (“services”) in the software development lifecycle, in order to meet cost, schedule, and time-to-market constraints. Testing, in particular, must scale to expected operational environment, plus provide support for critical non-functional requirements such as performance and security.
Automation in the software development process becomes critical as the scale of the IoT implementation rises. Security and non-functional requirements become more important as connectivity and scale increases. Service-based automated testing becomes critical during integration and system testing phases, while also enabling testing for security, stability, and performance.
Let’s take a look at Parasoft’s depth and breadth in test automation, as shown in Figure 1, as it applies to each phase of the SDLC. The key takeaway is how each solution complements the other and scales as the product grows. Unit testing is complemented by static and run-time analysis. During integration, unit testing progresses to API and service testing tools, which then progress to service virtualization.
Figure 1: Test automation solutions and where they are introduced into the software development lifecycle.
Most developers are familiar with some aspects of test automation; however, many are not thinking in terms of services or services-based testing and virtualization. As the paradigm shifts from “things” to “services,” the benefit of service-based testing becomes more apparent.
As a product progresses through the development process into integration, testing becomes more complicated and expensive. This is also the stage that many critical bugs are found, including security and performance issues not detectable at the unit level. A big risk is the availability of testing staff, labs, and hardware resources.
The solution here is automation. Automation provides a way to increase testing productivity, repeatability, and scope of system testing. An automated, services-based approach to testing is critical for the success of a newly-developed IoT device.
Benefits include the following:
A pure functional test isn’t enough to bring a device to market. Non-functional requirements such as security and performance are critical, but are some of the hardest characteristics to test for. A device that has poor performance or poor security simply isn’t competitive, but meeting time-to-market constraints and properly testing performance and security is a serious challenge for IoT devices. Service orientation provides a common way to specify requirements (e.g. performance on a per-service bases) and for testing security (e.g. attacks exploiting exposed services and APIs).
Performance and security requirements are likely to be expressed in terms of a quality-of-service statement. For example, an HVAC system might be required to maintain building temperature to 75 degrees within two hours based on a 5-degree difference in outside temperature, reporting current temperature every 10 seconds. If designing a thermostat for this product, you can test the functionality at the individual level and perhaps the performance too. However, if a deployed system consists of hundreds of thermostats, performance of the device is just a small part of a complex network of other devices.
Security requirements are often system-level and vague. In the HVAC system example, a thermostat might be required not to fail under heavy network loads. A denial-of-service attack relies on flooding the target with heavy traffic, sometimes with malformed packets. Penetration and fuzz testing is used in these cases to make sure devices can withstand a hostile network environment. Scalability available in a virtual environment exceeds what’s possible with real hardware.
The next logical step for automated service-based testing is virtualization. A complete virtual environment is possible by simulating all dependent components plus a full suite of test stimuli. The benefit of service virtualization is not just realistic and repeatable test environments but the ability to duplicate and deploy at will thus creating a “virtual lab” — a lab that’s just as effective as the real thing but at a fraction of the cost.
Test automation, in general, is critical to meeting IoT product goals such as time-to-market and budget. A service-based approach increases testing productivity, and allows for highly-repeatable and scalable tests for performance and security, along with the ability to build virtual test environments. As the complexity of IoT devices and their deployment environment grows, the need for scalable, service-level testing becomes more compelling.
Parasoft’s industry-leading automated software testing tools support the entire software development process, from when the developer writes the first line of code all the way through unit and functional testing, to performance and security testing, leveraging simulated test environments along the way.