Evaluation of a Delphi technique based expert judgement method for LCA valuation DELPHI II

Because of the complexity and trade-offs between different points of the life cycles of the analysed systems, a method which measures the environmental damage caused by each intervention is needed in order to make a choice between the products. However, there is no commonly agreed methodology for this particular purpose. In most of the methods the valuation is implicitly or explicitly based on economic criteria. For various reasons, however, economically obtained criteria do not necessarily reflect ecological arguments correctly. Thus, there is a need for new, ecologically based valuation methods. One such approach is the expert judgement method, based on the Delphi technique, which rejects the economic basis in favour of the judgements of a group of environmental experts. However, it is not self evident that the expert judgement based environmental rating of interventions will be essentially more correct and certain than other methods. In this study the method was evaluated at different points of the procedure in order to obtain a picture of the quality of the indexes produced. The evaluation was based on an actual Delphi study made in 1995-1996 in Finland, Sweden and Norway. The main questions addressed were the significance of the results and the operational quality of the Delphi procedure. The results obtained by applying the expert method indexes were also compared with the results obtained with other valuation methods for the background life cycle inventory of the case study. Additional material included feedback data from panellists of the case study, collected with a questionnaire. The questionnaire data was analysed to identify major dimensions in the criteria for evaluating interventions and correlation of the final indexes of the Delphi I study with these dimensions. The rest of the questionnaire material was used to document panellists' opinions and experiences of the Delphi process, familiarity with the environmental impacts of various interventions, and classification in typologies of cultural theory. The quality of results and methodological aspects, such as effects of task instructions, selection of the index basis, and effects of the final standardisation were analysed statistically. Accordingly, the effects of various postulates made on the conformity of the environmental harm conceptions of the experts, and the influence of the moderators' decisions were assessed on the basis of standard statistical indicators. The state of consensus and its development in the Delphi process were studied with the aid of K-entropy analysis. The study showed that transparency and certainty, which are essential qualities for an acceptable and trusted valuation method, are only partially accomplished by the expert judgement method in the format in which it was developed in the analysed case. As for the technical procedure, the method is well documented and transparency is good. Argumentation of the judgements, however, should be increased. The quality of the valuation indexes is explicitly available, but their certainty is very low for most interventions. The opinions of the experts vary greatly. How much this depends on different values and how much on differences in knowledge etc. is impossible to assess. Also, how much the technique used and the statistical processing of the experts' answers may have influenced the eventual scores of different interventions is difficult to assess. The application of expert judgement to LCA valuation is a new idea, and the method is still very much under development and far from maturity. Nevertheless, utilisation of expert knowledge can be a significant addition to model approaches to ecological impact assessment, which, because of the chaotic behaviour of ecosystems, are limited and uncertain in predicting the ecological consequences of interventions to the environment. This should be taken into account when considering the results of the evaluation of the case study, which was the third of its kind in Europe.