Software testing with code-based test generators: data and lessons learned from a case study with an industrial software component

Pietro Braione (Corresponding Author), Giovanni Denaro, Andrea Mattavelli, Mattia Vivanti, Ali Muhammad

    Research output: Contribution to journalArticleScientificpeer-review

    16 Citations (Scopus)

    Abstract

    Automatically generating effective test suites promises a significant impact on testing practice by promoting extensively tested software within reasonable effort and cost bounds. Code-based test generators rely on the source code of the software under test to identify test objectives and to steer the test case generation process accordingly. Currently, the most mature proposals on this topic come from the research on random testing, dynamic symbolic execution, and search-based testing. This paper studies the effectiveness of a set of state-of-the-research test generators on a family of industrial programs with nontrivial domain-specific peculiarities. These programs are part of a software component of a real-time and safety-critical control system and integrate in a control task specified in LabVIEW, a graphical language for designing embedded systems. The result of this study enhances the available body of knowledge on the strengths and weaknesses of test generators. The empirical data indicate that the test generators can truly expose subtle (previously unknown) bugs in the subject software and that there can be merit in using different types of test generation approaches in a complementary, even synergic fashion. Furthermore, our experiment pinpoints the support for floating point arithmetics and nonlinear computations as a major milestone in the path to exploiting the full potential of the prototypes based on symbolic execution in industry.
    Original languageEnglish
    Pages (from-to)311-333
    Number of pages23
    JournalSoftware Quality Journal
    Volume22
    Issue number2
    DOIs
    Publication statusPublished - 2014
    MoE publication typeA1 Journal article-refereed

    Keywords

    • structural testing
    • automatic test generation
    • experimental study

    Fingerprint Dive into the research topics of 'Software testing with code-based test generators: data and lessons learned from a case study with an industrial software component'. Together they form a unique fingerprint.

    Cite this