Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

Adrian Santos; Janne Järvinen; Jari Partanen; Markku Oivo; Natalia Juristo

doi:10.1007/978-3-030-03673-7_17

Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

Adrian Santos^* (Corresponding author), Janne Järvinen, Jari Partanen, Markku Oivo, Natalia Juristo

^*Corresponding author for this work

Not published at VTT

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

2 Citations (Scopus)

Abstract

Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across the premises of the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of TDD’s performance in realistic settings.

Original language	English
Title of host publication	Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings
Editors	Marcus Ciolkowski, Regina Hebig, Marco Kuhrmann, Dietmar Pfahl, Paolo Tell, Sousuke Amasaki, Steffen Küpper, Kurt Schneider, Jil Klünder
Publisher	Springer
Pages	227-242
Number of pages	16
ISBN (Electronic)	978-3-030-03673-7
ISBN (Print)	978-3-030-03672-0
DOIs	https://doi.org/10.1007/978-3-030-03673-7_17
Publication status	Published - 2018
MoE publication type	A4 Article in a conference publication
Event	19th International Conference on Product-Focused Software Process Improvement, PROFES 2018 - Wolfsburg, Germany Duration: 28 Nov 2018 → 30 Nov 2018

Publication series

Series	Lecture Notes in Computer Science
Volume	11271 LNCS
ISSN	0302-9743

Conference

Conference	19th International Conference on Product-Focused Software Process Improvement, PROFES 2018
Country/Territory	Germany
City	Wolfsburg
Period	28/11/18 → 30/11/18

Keywords

Company
Experiment
Industry
Quality
Test-Driven Development

Access to Document

10.1007/978-3-030-03673-7_17

Cite this

Santos, A., Järvinen, J., Partanen, J., Oivo, M., & Juristo, N. (2018). Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD. In M. Ciolkowski, R. Hebig, M. Kuhrmann, D. Pfahl, P. Tell, S. Amasaki, S. Küpper, K. Schneider, & J. Klünder (Eds.), Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings (pp. 227-242). Springer. https://doi.org/10.1007/978-3-030-03673-7_17

Santos, Adrian ; Järvinen, Janne ; Partanen, Jari et al. / Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD. Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings. editor / Marcus Ciolkowski ; Regina Hebig ; Marco Kuhrmann ; Dietmar Pfahl ; Paolo Tell ; Sousuke Amasaki ; Steffen Küpper ; Kurt Schneider ; Jil Klünder. Springer, 2018. pp. 227-242 (Lecture Notes in Computer Science, Vol. 11271 LNCS).

@inproceedings{fca36dd00b5c49eb8bccfc32e311f733,

title = "Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD",

abstract = "Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across the premises of the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of TDD{\textquoteright}s performance in realistic settings.",

keywords = "Company, Experiment, Industry, Quality, Test-Driven Development",

author = "Adrian Santos and Janne J{\"a}rvinen and Jari Partanen and Markku Oivo and Natalia Juristo",

note = "Funding Information: This research was developed with the support of the Spanish Ministry of Science and Innovation project TIN2014-60490-P. Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2018. Copyright: Copyright 2019 Elsevier B.V., All rights reserved.; 19th International Conference on Product-Focused Software Process Improvement, PROFES 2018 ; Conference date: 28-11-2018 Through 30-11-2018",

year = "2018",

doi = "10.1007/978-3-030-03673-7_17",

language = "English",

isbn = "978-3-030-03672-0",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "227--242",

editor = "Marcus Ciolkowski and Regina Hebig and Marco Kuhrmann and Dietmar Pfahl and Paolo Tell and Sousuke Amasaki and Steffen K{\"u}pper and Kurt Schneider and Jil Kl{\"u}nder",

booktitle = "Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings",

address = "Germany",

}

Santos, A, Järvinen, J, Partanen, J, Oivo, M & Juristo, N 2018, Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD. in M Ciolkowski, R Hebig, M Kuhrmann, D Pfahl, P Tell, S Amasaki, S Küpper, K Schneider & J Klünder (eds), Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings. Springer, Lecture Notes in Computer Science, vol. 11271 LNCS, pp. 227-242, 19th International Conference on Product-Focused Software Process Improvement, PROFES 2018, Wolfsburg, Germany, 28/11/18. https://doi.org/10.1007/978-3-030-03673-7_17

Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD. / Santos, Adrian (Corresponding author); Järvinen, Janne; Partanen, Jari et al.
Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings. ed. / Marcus Ciolkowski; Regina Hebig; Marco Kuhrmann; Dietmar Pfahl; Paolo Tell; Sousuke Amasaki; Steffen Küpper; Kurt Schneider; Jil Klünder. Springer, 2018. p. 227-242 (Lecture Notes in Computer Science, Vol. 11271 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

AU - Santos, Adrian

AU - Järvinen, Janne

AU - Partanen, Jari

AU - Oivo, Markku

AU - Juristo, Natalia

N1 - Funding Information: This research was developed with the support of the Spanish Ministry of Science and Innovation project TIN2014-60490-P. Publisher Copyright: © Springer Nature Switzerland AG 2018. Copyright: Copyright 2019 Elsevier B.V., All rights reserved.

PY - 2018

Y1 - 2018

N2 - Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across the premises of the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of TDD’s performance in realistic settings.

AB - Test-Driven Development (TDD) has been claimed to increase external software quality. However, the extent to which TDD increases external quality has been seldom studied in industrial experiments. We conduct four industrial experiments in two different companies to evaluate the performance of TDD on external quality. We study whether the performance of TDD holds across the premises of the same company and across companies. We identify participant-level characteristics impacting results. Iterative-Test Last (ITL), the reverse approach of TDD, outperforms TDD in three out of four premises. ITL outperforms TDD in both companies. The larger the experience with unit testing and testing tools, the larger the difference in performance between ITL and TDD (in favour of ITL). Technological environment (i.e., programming language and testing tool) seems not to impact results. Evaluating participant-level characteristics impacting results in industrial experiments may ease the understanding of TDD’s performance in realistic settings.

KW - Company

KW - Experiment

KW - Industry

KW - Quality

KW - Test-Driven Development

UR - http://www.scopus.com/inward/record.url?scp=85057242067&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-03673-7_17

DO - 10.1007/978-3-030-03673-7_17

M3 - Conference article in proceedings

AN - SCOPUS:85057242067

SN - 978-3-030-03672-0

T3 - Lecture Notes in Computer Science

SP - 227

EP - 242

BT - Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings

A2 - Ciolkowski, Marcus

A2 - Hebig, Regina

A2 - Kuhrmann, Marco

A2 - Pfahl, Dietmar

A2 - Tell, Paolo

A2 - Amasaki, Sousuke

A2 - Küpper, Steffen

A2 - Schneider, Kurt

A2 - Klünder, Jil

PB - Springer

T2 - 19th International Conference on Product-Focused Software Process Improvement, PROFES 2018

Y2 - 28 November 2018 through 30 November 2018

ER -

Santos A, Järvinen J, Partanen J, Oivo M, Juristo N. Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD. In Ciolkowski M, Hebig R, Kuhrmann M, Pfahl D, Tell P, Amasaki S, Küpper S, Schneider K, Klünder J, editors, Product-Focused Software Process Improvement - 19th International Conference, PROFES 2018, Proceedings. Springer. 2018. p. 227-242. (Lecture Notes in Computer Science, Vol. 11271 LNCS). doi: 10.1007/978-3-030-03673-7_17

Does the performance of TDD hold across software companies and premises? A group of industrial experiments on TDD

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this