Usability evaluation of complex systems. A literature review

In this literature study the task of usability evaluation is approached through recent publications and known practises. The scope of the reviewed literature and the overall problematic of evaluating complex systems are first defined. Then existing evaluation methods and tools and a set of design guidelines and standards is introduced. In the review the distinction between the evaluation of the design process and evaluation of the outcome of the design process is made. Also the existing criteria for usability is provided. These criteria can be used as measures in the evaluation procedures described in the previous chapter. Then, the problem of validity in the evaluation and validation procedures of complex cognitive systems is shortly discusses. The final chapter covers the conclusions of the literature review. There is a need for deepening the relationship between usage and design. Usage must be the driver also in the design of process technology, plant’s safety, and instrumentation and control as well as that of the design of man-machine interface. The collaborative activity is called activity-oriented design. In order to enable activity-oriented design we need to create design methods that drive the design activity into this direction. Criteria for evaluating what is good design are not available in present standards. Criteria for good tools and practices must consider human in a favourable role in the various tasks of operating an industrial plant as opposed to the view of human as a source of error. This is the way to create an optimal system, which takes advantage of the unique capabilities of both the human operators and the automation systems.