Supporting structure-based test design using model checking

Jussi Lahtinen

Research output: Book/ReportReportProfessional

Abstract

Nuclear domain safety systems are commonly designed using function block diagrams that are automatically translated into software code. These safety systems need to be rigorously verified. One of the verification techniques required by standards and nuclear regulators is structure-based testing. Structure-based testing of automatically generated code is not effective in detecting defects in function block diagrams. Hence, several approaches for structurebased testing on the level of the function block diagrams have recently emerged. We have defined three structure-based test criteria for function block diagrams, and developed an automatic technique for designing test cases according to these criteria that uses model checking to generate the test cases. Unlike other similar test criteria, the developed criteria especially focus on the time-dependent aspects of the test requirements. We have tested our technique on fictitious function block diagrams, and a set of vendor-specific real-world industrial function block diagrams. The fault detection capability of the method is analysed using mutation analysis. The results suggest that the developed technique is scalable to most nuclear domain safety systems. The average fault detection capability of the generated tests ranged from 90 % to 95 % in our experiments.
Original languageEnglish
PublisherVTT Technical Research Centre of Finland
Number of pages19
Publication statusPublished - 2015
MoE publication typeD4 Published development or research report or study

Publication series

NameResearch Report
PublisherVTT
VolumeVTT-R-04004-15

Fingerprint

Model checking
Security systems
Fault detection
Testing
Codes (standards)
Defects
Experiments

Keywords

  • structure-based testing
  • function block diagram
  • model checking
  • mutation analysis

Cite this

Lahtinen, J. (2015). Supporting structure-based test design using model checking. VTT Technical Research Centre of Finland. VTT Research Report, Vol.. VTT-R-04004-15
Lahtinen, Jussi. / Supporting structure-based test design using model checking. VTT Technical Research Centre of Finland, 2015. 19 p. (VTT Research Report, Vol. VTT-R-04004-15).
@book{8eae1dd489ef4c939f43b90027ea40d4,
title = "Supporting structure-based test design using model checking",
abstract = "Nuclear domain safety systems are commonly designed using function block diagrams that are automatically translated into software code. These safety systems need to be rigorously verified. One of the verification techniques required by standards and nuclear regulators is structure-based testing. Structure-based testing of automatically generated code is not effective in detecting defects in function block diagrams. Hence, several approaches for structurebased testing on the level of the function block diagrams have recently emerged. We have defined three structure-based test criteria for function block diagrams, and developed an automatic technique for designing test cases according to these criteria that uses model checking to generate the test cases. Unlike other similar test criteria, the developed criteria especially focus on the time-dependent aspects of the test requirements. We have tested our technique on fictitious function block diagrams, and a set of vendor-specific real-world industrial function block diagrams. The fault detection capability of the method is analysed using mutation analysis. The results suggest that the developed technique is scalable to most nuclear domain safety systems. The average fault detection capability of the generated tests ranged from 90 {\%} to 95 {\%} in our experiments.",
keywords = "structure-based testing, function block diagram, model checking, mutation analysis",
author = "Jussi Lahtinen",
note = "Project code: 102392",
year = "2015",
language = "English",
series = "Research Report",
publisher = "VTT Technical Research Centre of Finland",
address = "Finland",

}

Lahtinen, J 2015, Supporting structure-based test design using model checking. VTT Research Report, vol. VTT-R-04004-15, VTT Technical Research Centre of Finland.

Supporting structure-based test design using model checking. / Lahtinen, Jussi.

VTT Technical Research Centre of Finland, 2015. 19 p. (VTT Research Report, Vol. VTT-R-04004-15).

Research output: Book/ReportReportProfessional

TY - BOOK

T1 - Supporting structure-based test design using model checking

AU - Lahtinen, Jussi

N1 - Project code: 102392

PY - 2015

Y1 - 2015

N2 - Nuclear domain safety systems are commonly designed using function block diagrams that are automatically translated into software code. These safety systems need to be rigorously verified. One of the verification techniques required by standards and nuclear regulators is structure-based testing. Structure-based testing of automatically generated code is not effective in detecting defects in function block diagrams. Hence, several approaches for structurebased testing on the level of the function block diagrams have recently emerged. We have defined three structure-based test criteria for function block diagrams, and developed an automatic technique for designing test cases according to these criteria that uses model checking to generate the test cases. Unlike other similar test criteria, the developed criteria especially focus on the time-dependent aspects of the test requirements. We have tested our technique on fictitious function block diagrams, and a set of vendor-specific real-world industrial function block diagrams. The fault detection capability of the method is analysed using mutation analysis. The results suggest that the developed technique is scalable to most nuclear domain safety systems. The average fault detection capability of the generated tests ranged from 90 % to 95 % in our experiments.

AB - Nuclear domain safety systems are commonly designed using function block diagrams that are automatically translated into software code. These safety systems need to be rigorously verified. One of the verification techniques required by standards and nuclear regulators is structure-based testing. Structure-based testing of automatically generated code is not effective in detecting defects in function block diagrams. Hence, several approaches for structurebased testing on the level of the function block diagrams have recently emerged. We have defined three structure-based test criteria for function block diagrams, and developed an automatic technique for designing test cases according to these criteria that uses model checking to generate the test cases. Unlike other similar test criteria, the developed criteria especially focus on the time-dependent aspects of the test requirements. We have tested our technique on fictitious function block diagrams, and a set of vendor-specific real-world industrial function block diagrams. The fault detection capability of the method is analysed using mutation analysis. The results suggest that the developed technique is scalable to most nuclear domain safety systems. The average fault detection capability of the generated tests ranged from 90 % to 95 % in our experiments.

KW - structure-based testing

KW - function block diagram

KW - model checking

KW - mutation analysis

M3 - Report

T3 - Research Report

BT - Supporting structure-based test design using model checking

PB - VTT Technical Research Centre of Finland

ER -

Lahtinen J. Supporting structure-based test design using model checking. VTT Technical Research Centre of Finland, 2015. 19 p. (VTT Research Report, Vol. VTT-R-04004-15).