Towards a Framework for Improving Software Development Process Mediated with CMMI Goals and Agile Practices: Dissertation

Problems in software development mainly spring from the difficulty of establishing and stabilizing the requirements, the changeability of the software and interactive dependency of the software, hardware and human beings. A software development process consists of a set of empirical and 'best' practices in software development, together with organization and management that are needed for the software product implementation. Different process models, such as CMMI (Capability Maturity Model Integration), ISO 9001 and ISO 15504, have been developed in the last decade to support the assessment of software development processes. The main process model, examined in this thesis, is CMMI. This model was chosen as the focus of this research because it is a widely-used, beneficial approach for identifying the key weaknesses of a software development process which need immediate attention and improvement. Two of the key challenges of CMMI assessments are 1) overly heavy and time-consuming assessments and 2) the risk that the achievement of CMMI levels forces the developers to use more time writing documents than implementing the software product. The level of interest in the use of agile practices (focusing on practices such as eXtreme Programming and Scrum) has radically increased in software organizations. Practitioners argue that the adoption of agile software development methods can solve the organizational need for a more rapid and flexible software development process, and enable improved communication in changing market situations. A brief analysis of the empirical body of knowledge reveals, however, that there are also several challenges in interactive dependency management and communication between the actors of software development in an agile context. The objective of this study is to increase the understanding of how improvements can be made in the software development process, mediated with CMMI 'specific' goals and agile practices from communication perspective. This study is based on a series of case studies and data from 4 companies and 8 software development teams. To prove the importance of the improvement approach, this study starts with an evaluation of the agile practices in current use, using well established 'innovation of adoption' theories. The evaluation indicates that agile practices can achieve the subsequent assimilation stages differentially. The results also support the use of an adoption strategy, in which the needs of the teams are first defined before mapping the agile practice-based improvement solutions to the project level challenges. Although the iteration retrospectives provide a practical way for improving a software development process at team level, companies need mechanisms to constantly implement improvement initiatives and share knowledge of the process status also at organizational level. To meet this gap in the current empirical body of knowledge and research, a novel framework is presented in this study. The framework can be used 1) to identify the agile practices for a plan-driven software development process and 2) to assess the software development process in a lightweight manner against the CMMI goals and agile practices. To indicate the value of the created framework, it is important to collect empirical evidence on how agile practices actually affect communication in the software development process. This study applies coordination theory to confirm that the adoption of agile practices, such as sprint planning, an open office space, daily meetings and product backlogs improve the communication and management of requirements, features and project task dependencies in agile software development teams. Additionally, increased informal communication can in some cases decrease the need for upfront documentation in software development teams and, therefore, facilitate more productive software development than in previous plan driven situations.