Verification-aided regression testing

Fabrizio Pastore, Leonardo Mariani, Antti E.J. Hyvärinen, Grigory Fedyukovich, Natasha Sharygina, Stephan Sehestedt, Ali Muhammad

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    9 Citations (Scopus)

    Abstract

    In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.
    Original languageEnglish
    Title of host publicationProceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014
    PublisherAssociation for Computing Machinery ACM
    Pages37-48
    ISBN (Print)978-145032645-2
    DOIs
    Publication statusPublished - 2014
    MoE publication typeA4 Article in a conference publication
    Event23rd International Symposium on Software Testing and Analysis, ISSTA 2014 - San Jose, United States
    Duration: 21 Jul 201425 Jul 2014

    Conference

    Conference23rd International Symposium on Software Testing and Analysis, ISSTA 2014
    Abbreviated titleISSTA 2014
    CountryUnited States
    CitySan Jose
    Period21/07/1425/07/14

    Fingerprint

    Testing
    Model checking

    Keywords

    • dynamic analysis
    • model checking
    • regression testing

    Cite this

    Pastore, F., Mariani, L., Hyvärinen, A. E. J., Fedyukovich, G., Sharygina, N., Sehestedt, S., & Muhammad, A. (2014). Verification-aided regression testing. In Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014 (pp. 37-48). Association for Computing Machinery ACM. https://doi.org/10.1145/2610384.2610387
    Pastore, Fabrizio ; Mariani, Leonardo ; Hyvärinen, Antti E.J. ; Fedyukovich, Grigory ; Sharygina, Natasha ; Sehestedt, Stephan ; Muhammad, Ali. / Verification-aided regression testing. Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, 2014. pp. 37-48
    @inproceedings{d6a2904a920547e2b28b457ef41c3c77,
    title = "Verification-aided regression testing",
    abstract = "In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.",
    keywords = "dynamic analysis, model checking, regression testing",
    author = "Fabrizio Pastore and Leonardo Mariani and Hyv{\"a}rinen, {Antti E.J.} and Grigory Fedyukovich and Natasha Sharygina and Stephan Sehestedt and Ali Muhammad",
    year = "2014",
    doi = "10.1145/2610384.2610387",
    language = "English",
    isbn = "978-145032645-2",
    pages = "37--48",
    booktitle = "Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014",
    publisher = "Association for Computing Machinery ACM",
    address = "United States",

    }

    Pastore, F, Mariani, L, Hyvärinen, AEJ, Fedyukovich, G, Sharygina, N, Sehestedt, S & Muhammad, A 2014, Verification-aided regression testing. in Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, pp. 37-48, 23rd International Symposium on Software Testing and Analysis, ISSTA 2014, San Jose, United States, 21/07/14. https://doi.org/10.1145/2610384.2610387

    Verification-aided regression testing. / Pastore, Fabrizio; Mariani, Leonardo; Hyvärinen, Antti E.J.; Fedyukovich, Grigory; Sharygina, Natasha; Sehestedt, Stephan; Muhammad, Ali.

    Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, 2014. p. 37-48.

    Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

    TY - GEN

    T1 - Verification-aided regression testing

    AU - Pastore, Fabrizio

    AU - Mariani, Leonardo

    AU - Hyvärinen, Antti E.J.

    AU - Fedyukovich, Grigory

    AU - Sharygina, Natasha

    AU - Sehestedt, Stephan

    AU - Muhammad, Ali

    PY - 2014

    Y1 - 2014

    N2 - In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.

    AB - In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.

    KW - dynamic analysis

    KW - model checking

    KW - regression testing

    U2 - 10.1145/2610384.2610387

    DO - 10.1145/2610384.2610387

    M3 - Conference article in proceedings

    SN - 978-145032645-2

    SP - 37

    EP - 48

    BT - Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014

    PB - Association for Computing Machinery ACM

    ER -

    Pastore F, Mariani L, Hyvärinen AEJ, Fedyukovich G, Sharygina N, Sehestedt S et al. Verification-aided regression testing. In Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM. 2014. p. 37-48 https://doi.org/10.1145/2610384.2610387