Evaluating the achievements and impacts of EC framework programme transport projects

Anu Tuominen (Corresponding Author), Tuuli Järvi, Kirsi Hyytinen, Evangelos Mitsakis, Maria Eugenia López-Lambas, L. la Paix-Puello, van der Jan Waard, A. Binsted, A. Sitov

    Research output: Contribution to journalArticleScientificpeer-review

    Abstract

    Purpose: The purpose of this paper is to present what kind of elements and evaluation methods should be included into a framework for evaluating the achievements and impacts of transport projects supported in EC Framework Programmes (FP). Further, the paper discusses the possibilities of such an evaluation framework in producing recommendations regarding future transport research and policy objectives as well as mutual learning for the basis of strategic long term planning. Methods: The paper describes the two-dimensional evaluation methodology developed in the course of the FP7 METRONOME project. The dimensions are: (1) achievement of project objectives and targets in different levels and (2) research project impacts according to four impact groups. The methodology uses four complementary approaches in evaluation, namely evaluation matrices, coordinator questionnaires, lead user interviews and workshops. Results: Based on the methodology testing, with a sample of FP5 and FP6 projects, the main results relating to the rationale, implementation and achievements of FP projects is presented. In general, achievement of objectives in both FPs was good. Strongest impacts were identified within the impact group of management and co-ordination. Also scientific and end-user impacts of the projects were adequate, but wider societal impacts quite modest. Conclusions: The paper concludes with a discussion both on the theoretical and practical implications of the proposed methodology and by presenting some relevant future research needs.
    Original languageEnglish
    Pages (from-to)59-74
    JournalEuropean Transport Research Review
    Volume3
    Issue number2
    DOIs
    Publication statusPublished - 2011
    MoE publication typeA1 Journal article-refereed

    Fingerprint

    European Community
    evaluation
    Planning
    methodology
    Testing
    research project
    Group
    planning
    questionnaire
    interview
    management
    learning

    Keywords

    • Transport research projects
    • Evaluation of research programmes
    • Evaluation methods

    Cite this

    Tuominen, A., Järvi, T., Hyytinen, K., Mitsakis, E., López-Lambas, M. E., la Paix-Puello, L., ... Sitov, A. (2011). Evaluating the achievements and impacts of EC framework programme transport projects. European Transport Research Review, 3(2), 59-74. https://doi.org/10.1007/s12544-011-0048-3
    Tuominen, Anu ; Järvi, Tuuli ; Hyytinen, Kirsi ; Mitsakis, Evangelos ; López-Lambas, Maria Eugenia ; la Paix-Puello, L. ; Waard, van der Jan ; Binsted, A. ; Sitov, A. / Evaluating the achievements and impacts of EC framework programme transport projects. In: European Transport Research Review. 2011 ; Vol. 3, No. 2. pp. 59-74.
    @article{a949fec22fa047dd86bc0ad201a72b9d,
    title = "Evaluating the achievements and impacts of EC framework programme transport projects",
    abstract = "Purpose: The purpose of this paper is to present what kind of elements and evaluation methods should be included into a framework for evaluating the achievements and impacts of transport projects supported in EC Framework Programmes (FP). Further, the paper discusses the possibilities of such an evaluation framework in producing recommendations regarding future transport research and policy objectives as well as mutual learning for the basis of strategic long term planning. Methods: The paper describes the two-dimensional evaluation methodology developed in the course of the FP7 METRONOME project. The dimensions are: (1) achievement of project objectives and targets in different levels and (2) research project impacts according to four impact groups. The methodology uses four complementary approaches in evaluation, namely evaluation matrices, coordinator questionnaires, lead user interviews and workshops. Results: Based on the methodology testing, with a sample of FP5 and FP6 projects, the main results relating to the rationale, implementation and achievements of FP projects is presented. In general, achievement of objectives in both FPs was good. Strongest impacts were identified within the impact group of management and co-ordination. Also scientific and end-user impacts of the projects were adequate, but wider societal impacts quite modest. Conclusions: The paper concludes with a discussion both on the theoretical and practical implications of the proposed methodology and by presenting some relevant future research needs.",
    keywords = "Transport research projects, Evaluation of research programmes, Evaluation methods",
    author = "Anu Tuominen and Tuuli J{\"a}rvi and Kirsi Hyytinen and Evangelos Mitsakis and L{\'o}pez-Lambas, {Maria Eugenia} and {la Paix-Puello}, L. and Waard, {van der Jan} and A. Binsted and A. Sitov",
    year = "2011",
    doi = "10.1007/s12544-011-0048-3",
    language = "English",
    volume = "3",
    pages = "59--74",
    journal = "European Transport Research Review",
    issn = "1867-0717",
    publisher = "Springer",
    number = "2",

    }

    Tuominen, A, Järvi, T, Hyytinen, K, Mitsakis, E, López-Lambas, ME, la Paix-Puello, L, Waard, VDJ, Binsted, A & Sitov, A 2011, 'Evaluating the achievements and impacts of EC framework programme transport projects', European Transport Research Review, vol. 3, no. 2, pp. 59-74. https://doi.org/10.1007/s12544-011-0048-3

    Evaluating the achievements and impacts of EC framework programme transport projects. / Tuominen, Anu (Corresponding Author); Järvi, Tuuli; Hyytinen, Kirsi; Mitsakis, Evangelos; López-Lambas, Maria Eugenia; la Paix-Puello, L.; Waard, van der Jan; Binsted, A.; Sitov, A.

    In: European Transport Research Review, Vol. 3, No. 2, 2011, p. 59-74.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - Evaluating the achievements and impacts of EC framework programme transport projects

    AU - Tuominen, Anu

    AU - Järvi, Tuuli

    AU - Hyytinen, Kirsi

    AU - Mitsakis, Evangelos

    AU - López-Lambas, Maria Eugenia

    AU - la Paix-Puello, L.

    AU - Waard, van der Jan

    AU - Binsted, A.

    AU - Sitov, A.

    PY - 2011

    Y1 - 2011

    N2 - Purpose: The purpose of this paper is to present what kind of elements and evaluation methods should be included into a framework for evaluating the achievements and impacts of transport projects supported in EC Framework Programmes (FP). Further, the paper discusses the possibilities of such an evaluation framework in producing recommendations regarding future transport research and policy objectives as well as mutual learning for the basis of strategic long term planning. Methods: The paper describes the two-dimensional evaluation methodology developed in the course of the FP7 METRONOME project. The dimensions are: (1) achievement of project objectives and targets in different levels and (2) research project impacts according to four impact groups. The methodology uses four complementary approaches in evaluation, namely evaluation matrices, coordinator questionnaires, lead user interviews and workshops. Results: Based on the methodology testing, with a sample of FP5 and FP6 projects, the main results relating to the rationale, implementation and achievements of FP projects is presented. In general, achievement of objectives in both FPs was good. Strongest impacts were identified within the impact group of management and co-ordination. Also scientific and end-user impacts of the projects were adequate, but wider societal impacts quite modest. Conclusions: The paper concludes with a discussion both on the theoretical and practical implications of the proposed methodology and by presenting some relevant future research needs.

    AB - Purpose: The purpose of this paper is to present what kind of elements and evaluation methods should be included into a framework for evaluating the achievements and impacts of transport projects supported in EC Framework Programmes (FP). Further, the paper discusses the possibilities of such an evaluation framework in producing recommendations regarding future transport research and policy objectives as well as mutual learning for the basis of strategic long term planning. Methods: The paper describes the two-dimensional evaluation methodology developed in the course of the FP7 METRONOME project. The dimensions are: (1) achievement of project objectives and targets in different levels and (2) research project impacts according to four impact groups. The methodology uses four complementary approaches in evaluation, namely evaluation matrices, coordinator questionnaires, lead user interviews and workshops. Results: Based on the methodology testing, with a sample of FP5 and FP6 projects, the main results relating to the rationale, implementation and achievements of FP projects is presented. In general, achievement of objectives in both FPs was good. Strongest impacts were identified within the impact group of management and co-ordination. Also scientific and end-user impacts of the projects were adequate, but wider societal impacts quite modest. Conclusions: The paper concludes with a discussion both on the theoretical and practical implications of the proposed methodology and by presenting some relevant future research needs.

    KW - Transport research projects

    KW - Evaluation of research programmes

    KW - Evaluation methods

    U2 - 10.1007/s12544-011-0048-3

    DO - 10.1007/s12544-011-0048-3

    M3 - Article

    VL - 3

    SP - 59

    EP - 74

    JO - European Transport Research Review

    JF - European Transport Research Review

    SN - 1867-0717

    IS - 2

    ER -