Providing test quality feedback using static source code and automatic test suite metrics

N. Nagappan; L. Williams; J. Osborne; M. Vouk; Pekka Abrahamsson

doi:10.1109/ISSRE.2005.35

Providing test quality feedback using static source code and automatic test suite metrics

N. Nagappan, L. Williams, J. Osborne, M. Vouk, Pekka Abrahamsson

VTT Technical Research Centre of Finland

North Carolina State University

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

19 Citations (Scopus)

Abstract

A classic question in software development is "How much testing is enough?" Aside from dynamic coverage-based metrics, there are few measures that can be used to provide guidance on the quality of an automatic test suite as development proceeds. This paper utilizes the software testing and reliability early warning (STREW) static metric suite to provide a developer with indications of changes and additions to their automated unit test suite and code for added confidence that product quality will be high. Retrospective case studies to assess the utility of using the STREW metrics as a feedback mechanism were performed in academic, open source and industrial environments. The results indicate at statistically significant levels the ability of the STREW metrics to provide feedback on important attributes of an automatic test suite and corresponding code.

Original language	English
Title of host publication	The 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005)
Publisher	IEEE Institute of Electrical and Electronic Engineers
Pages	85-94
ISBN (Print)	0-7695-2482-6
DOIs	https://doi.org/10.1109/ISSRE.2005.35
Publication status	Published - 2005
MoE publication type	A4 Article in a conference publication
Event	16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005) - Chicago, United States Duration: 8 Nov 2005 → 11 Nov 2005

Conference

Conference	16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005)
Country/Territory	United States
City	Chicago
Period	8/11/05 → 11/11/05

Keywords

feedback
program testing
software metrics
software quality
software reliability
automatic test suite metrics
product quality
reliability early warning static metric suite
software development
software testing
static source code
test quality feedback

Access to Document

10.1109/ISSRE.2005.35

Cite this

@inproceedings{bf5495e1ebc946a588aa36f85e8aa8ac,

title = "Providing test quality feedback using static source code and automatic test suite metrics",

abstract = "A classic question in software development is {"}How much testing is enough?{"} Aside from dynamic coverage-based metrics, there are few measures that can be used to provide guidance on the quality of an automatic test suite as development proceeds. This paper utilizes the software testing and reliability early warning (STREW) static metric suite to provide a developer with indications of changes and additions to their automated unit test suite and code for added confidence that product quality will be high. Retrospective case studies to assess the utility of using the STREW metrics as a feedback mechanism were performed in academic, open source and industrial environments. The results indicate at statistically significant levels the ability of the STREW metrics to provide feedback on important attributes of an automatic test suite and corresponding code.",

keywords = "feedback, program testing, software metrics, software quality, software reliability, automatic test suite metrics, product quality, reliability early warning static metric suite, software development, software testing, static source code, test quality feedback",

author = "N. Nagappan and L. Williams and J. Osborne and M. Vouk and Pekka Abrahamsson",

year = "2005",

doi = "10.1109/ISSRE.2005.35",

language = "English",

isbn = "0-7695-2482-6",

pages = "85--94",

booktitle = "The 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005)",

publisher = "IEEE Institute of Electrical and Electronic Engineers",

address = "United States",

note = "16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005) ; Conference date: 08-11-2005 Through 11-11-2005",

}

Nagappan, N, Williams, L, Osborne, J, Vouk, M & Abrahamsson, P 2005, Providing test quality feedback using static source code and automatic test suite metrics. in The 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005). IEEE Institute of Electrical and Electronic Engineers, pp. 85-94, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005), Chicago, Illinois, United States, 8/11/05. https://doi.org/10.1109/ISSRE.2005.35

Providing test quality feedback using static source code and automatic test suite metrics. / Nagappan, N.; Williams, L.; Osborne, J. et al.
The 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005). IEEE Institute of Electrical and Electronic Engineers, 2005. p. 85-94.

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Providing test quality feedback using static source code and automatic test suite metrics

AU - Nagappan, N.

AU - Williams, L.

AU - Osborne, J.

AU - Vouk, M.

AU - Abrahamsson, Pekka

PY - 2005

Y1 - 2005

N2 - A classic question in software development is "How much testing is enough?" Aside from dynamic coverage-based metrics, there are few measures that can be used to provide guidance on the quality of an automatic test suite as development proceeds. This paper utilizes the software testing and reliability early warning (STREW) static metric suite to provide a developer with indications of changes and additions to their automated unit test suite and code for added confidence that product quality will be high. Retrospective case studies to assess the utility of using the STREW metrics as a feedback mechanism were performed in academic, open source and industrial environments. The results indicate at statistically significant levels the ability of the STREW metrics to provide feedback on important attributes of an automatic test suite and corresponding code.

AB - A classic question in software development is "How much testing is enough?" Aside from dynamic coverage-based metrics, there are few measures that can be used to provide guidance on the quality of an automatic test suite as development proceeds. This paper utilizes the software testing and reliability early warning (STREW) static metric suite to provide a developer with indications of changes and additions to their automated unit test suite and code for added confidence that product quality will be high. Retrospective case studies to assess the utility of using the STREW metrics as a feedback mechanism were performed in academic, open source and industrial environments. The results indicate at statistically significant levels the ability of the STREW metrics to provide feedback on important attributes of an automatic test suite and corresponding code.

KW - feedback

KW - program testing

KW - software metrics

KW - software quality

KW - software reliability

KW - automatic test suite metrics

KW - product quality

KW - reliability early warning static metric suite

KW - software development

KW - software testing

KW - static source code

KW - test quality feedback

U2 - 10.1109/ISSRE.2005.35

DO - 10.1109/ISSRE.2005.35

M3 - Conference article in proceedings

SN - 0-7695-2482-6

SP - 85

EP - 94

BT - The 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005)

PB - IEEE Institute of Electrical and Electronic Engineers

T2 - 16th IEEE International Symposium on Software Reliability Engineering (ISSRE 2005)

Y2 - 8 November 2005 through 11 November 2005

ER -

Providing test quality feedback using static source code and automatic test suite metrics

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this