Agile Evaluation Methods for Intelligent Transport Systems – Definitions of Agile Characteristics and Assessment Criteria

    Research output: ThesisDissertationCollection of Articles

    Abstract

    Information on the technical functioning, impacts, interoperability and acceptance of ITS systems is crucial to making decisions on the deployment of these systems. The resources, time and amount of data available for evaluation studies of a potential ITS system are often more or less limited. A need for more agile evaluation methods has been identified. This study provides definitions for different agile characteristics of evaluation methods for ITS systems, measurable criteria for these characteristics, and a definition for an agile evaluation method. The agile characteristics were defined on the basis of a literature study taking into account the Agile Manifesto, literature on measuring the agility of software development methods, and the experience obtained from seven case studies. The agile characteristics found to be relevant for evaluation methods for ITS include flexibility, leanness, lightness and simplicity, responsiveness, learning, speed and sustainability. The descriptions of agile characteristics, criteria for measuring the agility of evaluation methods and the definition of an agile evaluation method were validated by applying them to seven case studies. The case studies involved evaluations of technical functioning, impacts, socio-economic benefits, interoperability and user acceptance of ITS systems. Of the seven evaluated case studies, five were found to be based on agile methods as such or with restrictions. The study results indicated that agile methods for evaluating the impacts of ITS have many similar characteristics to expert assessment, such as limited requirements for time and resources. However, they can be expected to be more transparent and possibly also more accurate than expert assessment without supporting documentation. Agile methods for evaluating the technical functioning of ITS systems, based on observation of system behaviour, analysis of reliability and statistical analysis of study results, can be used to verify the operation of an ITS system or subsystem in realistic operating conditions. They can be applied also in situations in which no extensive up-to-date documentation on system requirements is produced or maintained, such as when an agile development model is used for developing an ITS system. The study results indicate that not all simple or simplistic evaluation methods are agile methods, as simplicity does not imply that the evaluation method has other essential characteristics of agility such as flexibility and leanness. More validation will likely be needed for descriptions of the agile characteristics of evaluation methods, for the criteria for agility, and for the definition of an agile evaluation method. Further analyses should also be carried out on how to employ agile development, testing and systems engineering methods in the development of ITS systems in an optimum way, and how agile evaluation methods could be connected to them.
    Original languageEnglish
    QualificationDoctor Degree
    Awarding Institution
    • Aalto University
    Supervisors/Advisors
    • Kulmala, Risto, Advisor, External person
    • Hämäläinen, Jyri, Supervisor, External person
    Award date26 Sep 2019
    Place of PublicationHelsinki
    Publisher
    Print ISBNs978-952-60-8666-8
    Electronic ISBNs978-952-60-8667-5
    Publication statusPublished - 2019
    MoE publication typeG5 Doctoral dissertation (article)

    Fingerprint

    Interoperability
    Reliability analysis
    Systems engineering
    Sustainable development
    Software engineering
    Statistical methods
    Decision making
    Economics
    Testing

    Keywords

    • intelligent transport systems
    • evaluation
    • agile
    • road safety
    • advanced driver assistance

    Cite this

    @phdthesis{e2de1077f76d4710bc6b3f11b4d46fa7,
    title = "Agile Evaluation Methods for Intelligent Transport Systems – Definitions of Agile Characteristics and Assessment Criteria",
    abstract = "Information on the technical functioning, impacts, interoperability and acceptance of ITS systems is crucial to making decisions on the deployment of these systems. The resources, time and amount of data available for evaluation studies of a potential ITS system are often more or less limited. A need for more agile evaluation methods has been identified. This study provides definitions for different agile characteristics of evaluation methods for ITS systems, measurable criteria for these characteristics, and a definition for an agile evaluation method. The agile characteristics were defined on the basis of a literature study taking into account the Agile Manifesto, literature on measuring the agility of software development methods, and the experience obtained from seven case studies. The agile characteristics found to be relevant for evaluation methods for ITS include flexibility, leanness, lightness and simplicity, responsiveness, learning, speed and sustainability. The descriptions of agile characteristics, criteria for measuring the agility of evaluation methods and the definition of an agile evaluation method were validated by applying them to seven case studies. The case studies involved evaluations of technical functioning, impacts, socio-economic benefits, interoperability and user acceptance of ITS systems. Of the seven evaluated case studies, five were found to be based on agile methods as such or with restrictions. The study results indicated that agile methods for evaluating the impacts of ITS have many similar characteristics to expert assessment, such as limited requirements for time and resources. However, they can be expected to be more transparent and possibly also more accurate than expert assessment without supporting documentation. Agile methods for evaluating the technical functioning of ITS systems, based on observation of system behaviour, analysis of reliability and statistical analysis of study results, can be used to verify the operation of an ITS system or subsystem in realistic operating conditions. They can be applied also in situations in which no extensive up-to-date documentation on system requirements is produced or maintained, such as when an agile development model is used for developing an ITS system. The study results indicate that not all simple or simplistic evaluation methods are agile methods, as simplicity does not imply that the evaluation method has other essential characteristics of agility such as flexibility and leanness. More validation will likely be needed for descriptions of the agile characteristics of evaluation methods, for the criteria for agility, and for the definition of an agile evaluation method. Further analyses should also be carried out on how to employ agile development, testing and systems engineering methods in the development of ITS systems in an optimum way, and how agile evaluation methods could be connected to them.",
    keywords = "intelligent transport systems, evaluation, agile, road safety, advanced driver assistance",
    author = "Risto {\"O}{\"o}rni",
    year = "2019",
    language = "English",
    isbn = "978-952-60-8666-8",
    series = "Aalto University Publication Series: Doctoral Dissertations",
    publisher = "Aalto University",
    number = "142",
    address = "Finland",
    school = "Aalto University",

    }

    Agile Evaluation Methods for Intelligent Transport Systems – Definitions of Agile Characteristics and Assessment Criteria. / Öörni, Risto.

    Helsinki : Aalto University, 2019. 194 p.

    Research output: ThesisDissertationCollection of Articles

    TY - THES

    T1 - Agile Evaluation Methods for Intelligent Transport Systems – Definitions of Agile Characteristics and Assessment Criteria

    AU - Öörni, Risto

    PY - 2019

    Y1 - 2019

    N2 - Information on the technical functioning, impacts, interoperability and acceptance of ITS systems is crucial to making decisions on the deployment of these systems. The resources, time and amount of data available for evaluation studies of a potential ITS system are often more or less limited. A need for more agile evaluation methods has been identified. This study provides definitions for different agile characteristics of evaluation methods for ITS systems, measurable criteria for these characteristics, and a definition for an agile evaluation method. The agile characteristics were defined on the basis of a literature study taking into account the Agile Manifesto, literature on measuring the agility of software development methods, and the experience obtained from seven case studies. The agile characteristics found to be relevant for evaluation methods for ITS include flexibility, leanness, lightness and simplicity, responsiveness, learning, speed and sustainability. The descriptions of agile characteristics, criteria for measuring the agility of evaluation methods and the definition of an agile evaluation method were validated by applying them to seven case studies. The case studies involved evaluations of technical functioning, impacts, socio-economic benefits, interoperability and user acceptance of ITS systems. Of the seven evaluated case studies, five were found to be based on agile methods as such or with restrictions. The study results indicated that agile methods for evaluating the impacts of ITS have many similar characteristics to expert assessment, such as limited requirements for time and resources. However, they can be expected to be more transparent and possibly also more accurate than expert assessment without supporting documentation. Agile methods for evaluating the technical functioning of ITS systems, based on observation of system behaviour, analysis of reliability and statistical analysis of study results, can be used to verify the operation of an ITS system or subsystem in realistic operating conditions. They can be applied also in situations in which no extensive up-to-date documentation on system requirements is produced or maintained, such as when an agile development model is used for developing an ITS system. The study results indicate that not all simple or simplistic evaluation methods are agile methods, as simplicity does not imply that the evaluation method has other essential characteristics of agility such as flexibility and leanness. More validation will likely be needed for descriptions of the agile characteristics of evaluation methods, for the criteria for agility, and for the definition of an agile evaluation method. Further analyses should also be carried out on how to employ agile development, testing and systems engineering methods in the development of ITS systems in an optimum way, and how agile evaluation methods could be connected to them.

    AB - Information on the technical functioning, impacts, interoperability and acceptance of ITS systems is crucial to making decisions on the deployment of these systems. The resources, time and amount of data available for evaluation studies of a potential ITS system are often more or less limited. A need for more agile evaluation methods has been identified. This study provides definitions for different agile characteristics of evaluation methods for ITS systems, measurable criteria for these characteristics, and a definition for an agile evaluation method. The agile characteristics were defined on the basis of a literature study taking into account the Agile Manifesto, literature on measuring the agility of software development methods, and the experience obtained from seven case studies. The agile characteristics found to be relevant for evaluation methods for ITS include flexibility, leanness, lightness and simplicity, responsiveness, learning, speed and sustainability. The descriptions of agile characteristics, criteria for measuring the agility of evaluation methods and the definition of an agile evaluation method were validated by applying them to seven case studies. The case studies involved evaluations of technical functioning, impacts, socio-economic benefits, interoperability and user acceptance of ITS systems. Of the seven evaluated case studies, five were found to be based on agile methods as such or with restrictions. The study results indicated that agile methods for evaluating the impacts of ITS have many similar characteristics to expert assessment, such as limited requirements for time and resources. However, they can be expected to be more transparent and possibly also more accurate than expert assessment without supporting documentation. Agile methods for evaluating the technical functioning of ITS systems, based on observation of system behaviour, analysis of reliability and statistical analysis of study results, can be used to verify the operation of an ITS system or subsystem in realistic operating conditions. They can be applied also in situations in which no extensive up-to-date documentation on system requirements is produced or maintained, such as when an agile development model is used for developing an ITS system. The study results indicate that not all simple or simplistic evaluation methods are agile methods, as simplicity does not imply that the evaluation method has other essential characteristics of agility such as flexibility and leanness. More validation will likely be needed for descriptions of the agile characteristics of evaluation methods, for the criteria for agility, and for the definition of an agile evaluation method. Further analyses should also be carried out on how to employ agile development, testing and systems engineering methods in the development of ITS systems in an optimum way, and how agile evaluation methods could be connected to them.

    KW - intelligent transport systems

    KW - evaluation

    KW - agile

    KW - road safety

    KW - advanced driver assistance

    M3 - Dissertation

    SN - 978-952-60-8666-8

    T3 - Aalto University Publication Series: Doctoral Dissertations

    PB - Aalto University

    CY - Helsinki

    ER -