Project description

Introduction

Software testing is a fundamental aspect of software development, and can consume as much as 50% of the total cost for a project. This is particularly true for real-time systems, which have to be rigorously tested to ensure their correctness. Further, real-time systems must not only be tested for logical correctness, but also for timeliness (which can be considered a functional property in a real-time system). A current trend in the requirements of distributed real-time systems is that complex operations must be supported while flexibility is preserved. Further, these system operate in environments where their correctness is of great importance. These characteristics imply the need of event-triggered real-time systems that is dependable. One method for increasing confidence in the dependability of real-time systems is testing. Unfortunately, current industry practice for testing real-time applications is often forced to resort to case-specific or ad-hoc techniques. One reason for this is that the execution order in distributed real-time systems is generally non-deterministic, due to dynamic scheduling and varying execution times.

Theoretical fundations of real-time testing

The determinism of real-time systems can be improved by applying constraints on the system architecture and the applications behavior, one example of this is to apply a purely time-triggered design paradigm. However, not all systems is designed under this paradigm, due to its inherent inflexibility.  Within this project we investigate the impacts of applying other types of architectural constraints, more suitable for event-triggered real-time systems with high flexibility requirements. The determinism and controllability of such system have impacts on system testability, even if this generally is not considerd in used defintions. We aim to generalize the testability concept so that system properties as well as program characteristics get reflected in the definition.

Framework for testing timeliness

In concurrent and event-triggered real-time systems the temporal correctness of the system depend not only on inputs to the application tasks but also on race-conditions and differences in execution times for concurrently executing tasks.  Testing of all anticipated internal behaviors (execution orders) is strongly desirable, but in general not feasible. Therefore, testing methods must be employed to select an subset of the system behaviors with high probability of revealing design flaws which leads to system failures. Within this project we propose and evaluate automatic test-case generation and execution methods for testing temporal correctness of flexible real-time systems.

Testing criteria for reliable real-time software

Even if only temporal correctness is a uniqe property of real-time systems, logical correctnes remain very important to test in these kind of systems. In particular, black-box techniques operating on the input domain are interesting to apply on the system level as well as on the unit testing levels. Therefore we analyze and evaluate capabilities of applying such test methods to, for example, real-time software within this project. In particular, testing methods based on covering combination of input and configuration parameters effecitvly, a.k.a. combination strategies, are investigated. These methods are simple to use and understand. Further, they provide meaningful coverage criteria and initial studies indicate that these methods may be effective in finding a large number of faults. Our main aim is to automate as  much as possible of the process of identifying, generating, and executing test cases based on combination strategies.

Support for model-based development for high testability

Design flaws are in general cheaper to correct if they are detected in the earlier phases of system development. This motivates the use of formal methods, which is a fault avoidance technique that complements testing. Our intention is to take full advantage of formal methods, not only the benefits of analysis and model checking coming from the use of a formal specification, but also for automatic code generation for timely transaction based applications.


Uppdaterad: 2008-11-27
Sidansvarig: DRTS-gruppen