Verification-aided regression testing

Fabrizio Pastore, Leonardo Mariani, Antti E.J. Hyvärinen, Grigory Fedyukovich, Natasha Sharygina, Stephan Sehestedt, Ali Muhammad

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

9 Citations (Scopus)

Abstract

In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.
Original languageEnglish
Title of host publicationProceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014
PublisherAssociation for Computing Machinery ACM
Pages37-48
ISBN (Print)978-145032645-2
DOIs
Publication statusPublished - 2014
MoE publication typeA4 Article in a conference publication
Event23rd International Symposium on Software Testing and Analysis, ISSTA 2014 - San Jose, United States
Duration: 21 Jul 201425 Jul 2014

Conference

Conference23rd International Symposium on Software Testing and Analysis, ISSTA 2014
Abbreviated titleISSTA 2014
CountryUnited States
CitySan Jose
Period21/07/1425/07/14

Fingerprint

Testing
Model checking

Keywords

  • dynamic analysis
  • model checking
  • regression testing

Cite this

Pastore, F., Mariani, L., Hyvärinen, A. E. J., Fedyukovich, G., Sharygina, N., Sehestedt, S., & Muhammad, A. (2014). Verification-aided regression testing. In Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014 (pp. 37-48). Association for Computing Machinery ACM. https://doi.org/10.1145/2610384.2610387
Pastore, Fabrizio ; Mariani, Leonardo ; Hyvärinen, Antti E.J. ; Fedyukovich, Grigory ; Sharygina, Natasha ; Sehestedt, Stephan ; Muhammad, Ali. / Verification-aided regression testing. Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, 2014. pp. 37-48
@inproceedings{d6a2904a920547e2b28b457ef41c3c77,
title = "Verification-aided regression testing",
abstract = "In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.",
keywords = "dynamic analysis, model checking, regression testing",
author = "Fabrizio Pastore and Leonardo Mariani and Hyv{\"a}rinen, {Antti E.J.} and Grigory Fedyukovich and Natasha Sharygina and Stephan Sehestedt and Ali Muhammad",
year = "2014",
doi = "10.1145/2610384.2610387",
language = "English",
isbn = "978-145032645-2",
pages = "37--48",
booktitle = "Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014",
publisher = "Association for Computing Machinery ACM",
address = "United States",

}

Pastore, F, Mariani, L, Hyvärinen, AEJ, Fedyukovich, G, Sharygina, N, Sehestedt, S & Muhammad, A 2014, Verification-aided regression testing. in Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, pp. 37-48, 23rd International Symposium on Software Testing and Analysis, ISSTA 2014, San Jose, United States, 21/07/14. https://doi.org/10.1145/2610384.2610387

Verification-aided regression testing. / Pastore, Fabrizio; Mariani, Leonardo; Hyvärinen, Antti E.J.; Fedyukovich, Grigory; Sharygina, Natasha; Sehestedt, Stephan; Muhammad, Ali.

Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM, 2014. p. 37-48.

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

TY - GEN

T1 - Verification-aided regression testing

AU - Pastore, Fabrizio

AU - Mariani, Leonardo

AU - Hyvärinen, Antti E.J.

AU - Fedyukovich, Grigory

AU - Sharygina, Natasha

AU - Sehestedt, Stephan

AU - Muhammad, Ali

PY - 2014

Y1 - 2014

N2 - In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.

AB - In this paper we present Verification-Aided Regression Testing (VART), a novel extension of regression testing that uses model checking to increase the fault revealing capability of existing test suites. The key idea in VART is to extend the use of test case executions from the conventional direct fault discovery to the generation of behavioral properties specific to the upgrade, by (i) automatically producing properties that are proved to hold for the base version of a program, (ii) automatically identifying and checking on the upgraded program only the properties that, according to the developers' intention, must be preserved by the upgrade, and (iii) reporting the faults and the corresponding counter-examples that are not revealed by the regression tests. Our empirical study on both open source and industrial software systems shows that VART automatically produces properties that increase the effectiveness of testing by automatically detecting faults unnoticed by the existing regression test suites.

KW - dynamic analysis

KW - model checking

KW - regression testing

U2 - 10.1145/2610384.2610387

DO - 10.1145/2610384.2610387

M3 - Conference article in proceedings

SN - 978-145032645-2

SP - 37

EP - 48

BT - Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014

PB - Association for Computing Machinery ACM

ER -

Pastore F, Mariani L, Hyvärinen AEJ, Fedyukovich G, Sharygina N, Sehestedt S et al. Verification-aided regression testing. In Proceedings of the 2014 International Symposium on Software Testing and Analysis, ISSTA 2014. Association for Computing Machinery ACM. 2014. p. 37-48 https://doi.org/10.1145/2610384.2610387