Research with an impact: Evaluation practices in public research organisations

This publication highlights the benefits of and the challenges faced in assessing the societal impacts of public research organisations, based on a research project (JYVA) focusing on the perceptions, practises and philosophies associated with the assessment of the socio-economic impacts of research and development. The JYVA-project was undertaken in a consortium between five Finnish public research organisations (PROs) involved in R&D activity: Agrifood Research Finland (MTT), Finnish Defence Forces Technical Research Centre (PVTT), Helia University of Business and Applied Sciences (HELIA), Satakunta University of Applied Sciences (SAMK) and the Technical Research Centre of Finland (VTT). The main aims of the project were to develop the necessary methods and indicators needed to analyse the impacts in each organisation, and to test an analytical framework for impact assessment developed within VTT. The main findings and policy implications are discussed here on the basis of three related perspectives: objectives (as they relate to organisational perspectives) indicators (as they relate to the innovation process more generally) learning (both in terms of actors and policy). The positive impacts that public research organizations generate originate in networks and with partners. Future competence development therefore requires an approach where organisational and cognitive needs relating to impact assessment are related specifically to the creation of new partnerships and network management, as well as to the role and "value added" of the organisation in these networks. On the level of policies, the role of the R&D organizations is essential in providing a cognitive base for developing cross-sectoral and more broadly based policy initiatives. In this regard management by results and management by programming are mutually supporting processes, where PROs should play an active role. Impact assessment helps to identify the strengths and weaknesses of organisational performance, as well as enabling us to focus on the key competences of the R&D organisations. By so doing it helps us to analyse the "additionality" of public R&D, from the point of view of current and potential customers, stakeholders and society at large. It is important to balance these external customer needs with the internal expertise development needs of the PROs however, and to do this in such a way that it does not jeopardise the creativity of individuals and institutional capabilities in expert organisations, as these are the core producers of continuous knowledge creation, acquisition, and transfer, ultimately ensuring continuing innovation into the future.