Research with an impact

Evaluation practices in public research organisations

Kaisa Lähteenmäki-Smith, Kirsi-Maria Hyytinen, Pirjo Kutinlahti, Jari Konttinen

Research output: Book/ReportReportProfessional

Abstract

This publication highlights the benefits of and the challenges faced in assessing the societal impacts of public research organisations, based on a research project (JYVA) focusing on the perceptions, practises and philosophies associated with the assessment of the socio-economic impacts of research and development. The JYVA-project was undertaken in a consortium between five Finnish public research organisations (PROs) involved in R&D activity: Agrifood Research Finland (MTT), Finnish Defence Forces Technical Research Centre (PVTT), Helia University of Business and Applied Sciences (HELIA), Satakunta University of Applied Sciences (SAMK) and the Technical Research Centre of Finland (VTT). The main aims of the project were to develop the necessary methods and indicators needed to analyse the impacts in each organisation, and to test an analytical framework for impact assessment developed within VTT. The main findings and policy implications are discussed here on the basis of three related perspectives: objectives (as they relate to organisational perspectives) indicators (as they relate to the innovation process more generally) learning (both in terms of actors and policy). The positive impacts that public research organizations generate originate in networks and with partners. Future competence development therefore requires an approach where organisational and cognitive needs relating to impact assessment are related specifically to the creation of new partnerships and network management, as well as to the role and "value added" of the organisation in these networks. On the level of policies, the role of the R&D organizations is essential in providing a cognitive base for developing cross-sectoral and more broadly based policy initiatives. In this regard management by results and management by programming are mutually supporting processes, where PROs should play an active role. Impact assessment helps to identify the strengths and weaknesses of organisational performance, as well as enabling us to focus on the key competences of the R&D organisations. By so doing it helps us to analyse the "additionality" of public R&D, from the point of view of current and potential customers, stakeholders and society at large. It is important to balance these external customer needs with the internal expertise development needs of the PROs however, and to do this in such a way that it does not jeopardise the creativity of individuals and institutional capabilities in expert organisations, as these are the core producers of continuous knowledge creation, acquisition, and transfer, ultimately ensuring continuing innovation into the future.
Original languageEnglish
Place of PublicationEspoo
PublisherVTT Technical Research Centre of Finland
Number of pages79
ISBN (Electronic)951-38-6785-4
ISBN (Print)951-38-6784-6
Publication statusPublished - 2006
MoE publication typeNot Eligible

Publication series

NameVTT Tiedotteita - Research Notes
PublisherVTT
No.2336
ISSN (Print)1235-0605
ISSN (Electronic)1455-0865

Fingerprint

Evaluation practice
Impact evaluation
Public research
Research organization
Impact assessment
Research center
Finland
Programming
Research activities
Innovation
Consortium
Competence development
Stakeholders
Customer needs
Innovation process
Additionality
External balance
Value added
Policy implications
Organizational performance

Keywords

  • evaluation
  • impact assessment
  • societal impacts
  • R&D
  • public research organisations (PROs)
  • polytechnics

Cite this

Lähteenmäki-Smith, K., Hyytinen, K-M., Kutinlahti, P., & Konttinen, J. (2006). Research with an impact: Evaluation practices in public research organisations. Espoo: VTT Technical Research Centre of Finland. VTT Tiedotteita - Meddelanden - Research Notes, No. 2336
Lähteenmäki-Smith, Kaisa ; Hyytinen, Kirsi-Maria ; Kutinlahti, Pirjo ; Konttinen, Jari. / Research with an impact : Evaluation practices in public research organisations. Espoo : VTT Technical Research Centre of Finland, 2006. 79 p. (VTT Tiedotteita - Meddelanden - Research Notes; No. 2336).
@book{e81a9d8c535840e7ad4d94cf94b652e5,
title = "Research with an impact: Evaluation practices in public research organisations",
abstract = "This publication highlights the benefits of and the challenges faced in assessing the societal impacts of public research organisations, based on a research project (JYVA) focusing on the perceptions, practises and philosophies associated with the assessment of the socio-economic impacts of research and development. The JYVA-project was undertaken in a consortium between five Finnish public research organisations (PROs) involved in R&D activity: Agrifood Research Finland (MTT), Finnish Defence Forces Technical Research Centre (PVTT), Helia University of Business and Applied Sciences (HELIA), Satakunta University of Applied Sciences (SAMK) and the Technical Research Centre of Finland (VTT). The main aims of the project were to develop the necessary methods and indicators needed to analyse the impacts in each organisation, and to test an analytical framework for impact assessment developed within VTT. The main findings and policy implications are discussed here on the basis of three related perspectives: objectives (as they relate to organisational perspectives) indicators (as they relate to the innovation process more generally) learning (both in terms of actors and policy). The positive impacts that public research organizations generate originate in networks and with partners. Future competence development therefore requires an approach where organisational and cognitive needs relating to impact assessment are related specifically to the creation of new partnerships and network management, as well as to the role and {"}value added{"} of the organisation in these networks. On the level of policies, the role of the R&D organizations is essential in providing a cognitive base for developing cross-sectoral and more broadly based policy initiatives. In this regard management by results and management by programming are mutually supporting processes, where PROs should play an active role. Impact assessment helps to identify the strengths and weaknesses of organisational performance, as well as enabling us to focus on the key competences of the R&D organisations. By so doing it helps us to analyse the {"}additionality{"} of public R&D, from the point of view of current and potential customers, stakeholders and society at large. It is important to balance these external customer needs with the internal expertise development needs of the PROs however, and to do this in such a way that it does not jeopardise the creativity of individuals and institutional capabilities in expert organisations, as these are the core producers of continuous knowledge creation, acquisition, and transfer, ultimately ensuring continuing innovation into the future.",
keywords = "evaluation, impact assessment, societal impacts, R&D, public research organisations (PROs), polytechnics",
author = "Kaisa L{\"a}hteenm{\"a}ki-Smith and Kirsi-Maria Hyytinen and Pirjo Kutinlahti and Jari Konttinen",
year = "2006",
language = "English",
isbn = "951-38-6784-6",
series = "VTT Tiedotteita - Research Notes",
publisher = "VTT Technical Research Centre of Finland",
number = "2336",
address = "Finland",

}

Lähteenmäki-Smith, K, Hyytinen, K-M, Kutinlahti, P & Konttinen, J 2006, Research with an impact: Evaluation practices in public research organisations. VTT Tiedotteita - Meddelanden - Research Notes, no. 2336, VTT Technical Research Centre of Finland, Espoo.

Research with an impact : Evaluation practices in public research organisations. / Lähteenmäki-Smith, Kaisa; Hyytinen, Kirsi-Maria; Kutinlahti, Pirjo; Konttinen, Jari.

Espoo : VTT Technical Research Centre of Finland, 2006. 79 p. (VTT Tiedotteita - Meddelanden - Research Notes; No. 2336).

Research output: Book/ReportReportProfessional

TY - BOOK

T1 - Research with an impact

T2 - Evaluation practices in public research organisations

AU - Lähteenmäki-Smith, Kaisa

AU - Hyytinen, Kirsi-Maria

AU - Kutinlahti, Pirjo

AU - Konttinen, Jari

PY - 2006

Y1 - 2006

N2 - This publication highlights the benefits of and the challenges faced in assessing the societal impacts of public research organisations, based on a research project (JYVA) focusing on the perceptions, practises and philosophies associated with the assessment of the socio-economic impacts of research and development. The JYVA-project was undertaken in a consortium between five Finnish public research organisations (PROs) involved in R&D activity: Agrifood Research Finland (MTT), Finnish Defence Forces Technical Research Centre (PVTT), Helia University of Business and Applied Sciences (HELIA), Satakunta University of Applied Sciences (SAMK) and the Technical Research Centre of Finland (VTT). The main aims of the project were to develop the necessary methods and indicators needed to analyse the impacts in each organisation, and to test an analytical framework for impact assessment developed within VTT. The main findings and policy implications are discussed here on the basis of three related perspectives: objectives (as they relate to organisational perspectives) indicators (as they relate to the innovation process more generally) learning (both in terms of actors and policy). The positive impacts that public research organizations generate originate in networks and with partners. Future competence development therefore requires an approach where organisational and cognitive needs relating to impact assessment are related specifically to the creation of new partnerships and network management, as well as to the role and "value added" of the organisation in these networks. On the level of policies, the role of the R&D organizations is essential in providing a cognitive base for developing cross-sectoral and more broadly based policy initiatives. In this regard management by results and management by programming are mutually supporting processes, where PROs should play an active role. Impact assessment helps to identify the strengths and weaknesses of organisational performance, as well as enabling us to focus on the key competences of the R&D organisations. By so doing it helps us to analyse the "additionality" of public R&D, from the point of view of current and potential customers, stakeholders and society at large. It is important to balance these external customer needs with the internal expertise development needs of the PROs however, and to do this in such a way that it does not jeopardise the creativity of individuals and institutional capabilities in expert organisations, as these are the core producers of continuous knowledge creation, acquisition, and transfer, ultimately ensuring continuing innovation into the future.

AB - This publication highlights the benefits of and the challenges faced in assessing the societal impacts of public research organisations, based on a research project (JYVA) focusing on the perceptions, practises and philosophies associated with the assessment of the socio-economic impacts of research and development. The JYVA-project was undertaken in a consortium between five Finnish public research organisations (PROs) involved in R&D activity: Agrifood Research Finland (MTT), Finnish Defence Forces Technical Research Centre (PVTT), Helia University of Business and Applied Sciences (HELIA), Satakunta University of Applied Sciences (SAMK) and the Technical Research Centre of Finland (VTT). The main aims of the project were to develop the necessary methods and indicators needed to analyse the impacts in each organisation, and to test an analytical framework for impact assessment developed within VTT. The main findings and policy implications are discussed here on the basis of three related perspectives: objectives (as they relate to organisational perspectives) indicators (as they relate to the innovation process more generally) learning (both in terms of actors and policy). The positive impacts that public research organizations generate originate in networks and with partners. Future competence development therefore requires an approach where organisational and cognitive needs relating to impact assessment are related specifically to the creation of new partnerships and network management, as well as to the role and "value added" of the organisation in these networks. On the level of policies, the role of the R&D organizations is essential in providing a cognitive base for developing cross-sectoral and more broadly based policy initiatives. In this regard management by results and management by programming are mutually supporting processes, where PROs should play an active role. Impact assessment helps to identify the strengths and weaknesses of organisational performance, as well as enabling us to focus on the key competences of the R&D organisations. By so doing it helps us to analyse the "additionality" of public R&D, from the point of view of current and potential customers, stakeholders and society at large. It is important to balance these external customer needs with the internal expertise development needs of the PROs however, and to do this in such a way that it does not jeopardise the creativity of individuals and institutional capabilities in expert organisations, as these are the core producers of continuous knowledge creation, acquisition, and transfer, ultimately ensuring continuing innovation into the future.

KW - evaluation

KW - impact assessment

KW - societal impacts

KW - R&D

KW - public research organisations (PROs)

KW - polytechnics

M3 - Report

SN - 951-38-6784-6

T3 - VTT Tiedotteita - Research Notes

BT - Research with an impact

PB - VTT Technical Research Centre of Finland

CY - Espoo

ER -

Lähteenmäki-Smith K, Hyytinen K-M, Kutinlahti P, Konttinen J. Research with an impact: Evaluation practices in public research organisations. Espoo: VTT Technical Research Centre of Finland, 2006. 79 p. (VTT Tiedotteita - Meddelanden - Research Notes; No. 2336).