Experimentally evaluating the dependability of complex software systems

Author: Lena Feinbube, Andreas Polze

This research focusses on evaluating software dependability both from a theoretical, and a practical perspective. 
Fault injection is a useful tool for practical experimentation. It can be used to test the fault tolerance features by artificially inserting faulty behaviour and error states into a running system. Thus, fault injection represents the notion that the faultload needs to be incorporated into the software testing process.
To increase representativeness and coverage of fault injection, the questions of when and where to inject faults need to be answered.
Therefore, the theoretical focus of this research is to better understand the details of fault activation and error propagation in software. Fault models for software are currently limited, either to purely code-based aspects, or to high level failure behaviour such as "crash fault". The state conditions which trigger the activation of dormant software faults, and thus lead to error states still form an elusive aspect.
A dependable software system must be fault tolerant at all layers of the software stack.
Our practical research focus lies upon fault injection at the operating system and distributed middleware layers.
At the operating system level, we show how different classes of software faults in libraries can be simulated using link-time API hooking. The fault classes are chosen as representatives from the community-maintained CWE database of real world software problems.
At the distributed level, modern cloud services have to fulfil needs such as scalability, availability and security simultaneously. Therefore, experimental dependability assessment using distributed fault injection is steadily gaining relevance. On-going research on fault injection in OpenStack is discussed.