A framework for observation-based modelling in model-based testing

TY - THES

T1 - A framework for observation-based modelling in model-based testing

T2 - Dissertation

AU - Kanstren, Teemu

N1 - Project code: 33451

PY - 2010

Y1 - 2010

N2 - In the context of software engineering, test automation as a field of research has been around for a very long time. Yet, testing and related concepts are still generally considered to be one of the most time-consuming and expensive parts of the software life cycle. Although it is a field with a relatively long research background, many existing test automation systems are still relatively simple and not very different from the early days. They still focus on executing an existing, usually manually crafted, set of tests over and over again. One approach that has also been around for a relatively long time but has only recently begun to attract considerable interest in the domain of software testing is model-based testing. In model-based testing, the system under test is represented by a model describing its expected behaviour at a higher abstraction level, and a set of chosen algorithms are used to generate tests from this model. Currently, these models need to be manually crafted from the specification. This thesis presents an approach for observation-based modelling in model-based testing and aims to provide automated assistance for model creation. This includes design and architectural solutions to support observation and testing of the system, analysis of different types of executions used as a basis for observations, and finally combines the different viewpoints to provide automated tool support to generate an initial test model, based on the captured observations, that is suitable for use in model-based testing. This model is then refined and verified against the specification. As the approach reverses the traditional model-based testing approach of going from specification to implementation, to going from implementation to specification, guidelines for its application are also presented. The research uses a constructive approach, in which a problem is identified, a construct to address the problem is designed and implemented, and finally the results are evaluated. The approach has been evaluated in the context of a practical system in which its application discovered several previously unknown bugs in the implementation of the system under test. Its effectiveness was also demonstrated by generating a highly complete model and showing how the completed model provides additional test coverage both in terms of code covered and injected faults discovered (test mutants killed).

AB - In the context of software engineering, test automation as a field of research has been around for a very long time. Yet, testing and related concepts are still generally considered to be one of the most time-consuming and expensive parts of the software life cycle. Although it is a field with a relatively long research background, many existing test automation systems are still relatively simple and not very different from the early days. They still focus on executing an existing, usually manually crafted, set of tests over and over again. One approach that has also been around for a relatively long time but has only recently begun to attract considerable interest in the domain of software testing is model-based testing. In model-based testing, the system under test is represented by a model describing its expected behaviour at a higher abstraction level, and a set of chosen algorithms are used to generate tests from this model. Currently, these models need to be manually crafted from the specification. This thesis presents an approach for observation-based modelling in model-based testing and aims to provide automated assistance for model creation. This includes design and architectural solutions to support observation and testing of the system, analysis of different types of executions used as a basis for observations, and finally combines the different viewpoints to provide automated tool support to generate an initial test model, based on the captured observations, that is suitable for use in model-based testing. This model is then refined and verified against the specification. As the approach reverses the traditional model-based testing approach of going from specification to implementation, to going from implementation to specification, guidelines for its application are also presented. The research uses a constructive approach, in which a problem is identified, a construct to address the problem is designed and implemented, and finally the results are evaluated. The approach has been evaluated in the context of a practical system in which its application discovered several previously unknown bugs in the implementation of the system under test. Its effectiveness was also demonstrated by generating a highly complete model and showing how the completed model provides additional test coverage both in terms of code covered and injected faults discovered (test mutants killed).

KW - model-based testing

KW - test automation

KW - observation-based modelling

KW - test generation

M3 - Dissertation

SN - 978-951-38-7376-9

T3 - VTT Publications

PB - VTT Technical Research Centre of Finland

CY - Espoo

ER -