Evaluating systems usability in complex work

TY - THES

T1 - Evaluating systems usability in complex work

T2 - Development of a systemic usability concept to benefit control room design: Dissertation

AU - Savioja, Paula

PY - 2014

Y1 - 2014

N2 - The design of industrial control rooms assumes fulfilling the goals of production, safety, and human well-being. Control rooms and the user interfaces within them should enable the effective and efficient conduct of work in all the operating conditions which can be foreseen during specification and design. At the same time, the user interfaces should enable the control of the process in unprecedented and totally unexpected situations while simultaneously maintaining the safety of the process. During the design or modification of the control room, the potentiality of the emerging solution to fulfil the objectives is assessed by conducting empirical evaluations. This dissertation presents the development of an evaluation methodology which enables developing the control room towards meeting these objectives. Industrial process control constitutes a socio-technical system in which people and technologies have multiple and sometimes overlapping roles. In order to meet the demands of maintaining safety in all situations, the socio-technical system should have built-in capability of dealing with the unexpected. The control room, and the user interfaces within it, are an integral part of the socio-technical system. Thus, they have a role in construing and maintaining the safety of the system. The concept of systems usability (SU) is introduced in the dissertation to evaluate the systemic effects of control room solutions. SU is a human-centred quality attribute of user interfaces and control rooms attributed to technology, but the value in the use of the technology is evidenced in the success of the activity in which the technology is used. Thus, the research makes sense of the significance of the individual technological solutions in, and for, the entirety of an activity system. Systems usability means that a tool in an activity serves the functions of 1) an instrument, 2) a psychological tool, and 3) a communicative tool. The meaning of each function in the specific domain is contextually defined. Furthermore, the quality of the tool can be assessed utilising different perspectives on the usage activity: performance, way of acting, and user experience. By combining the functions of the tool and perspectives on activity, a systemic framework for developing contextual indicators for a good control room is construed. Utilising the concept of SU in the control room requires a model-based evaluation approach. This means that the general contextual work demands are considered in defining the reference of a successful process control activity. In addition, the scenarios utilised in the evaluation are modelled also taking the general work demands into account. A specific scenario modelling method, functional situation modelling (FSM), is presented in this dissertation. FSM combines a functional and a chronological view of the activity of an operating crew in a particular situation. By making explicit the connection between required operating actions and critical domain functions, the model lays the ground for analysing operating activity from the point of view of maintaining the critical functions. On aspect of control room evaluation is identifying whether the operating practices are attuned to maintaining safety in all situations. For this purpose, a data analysis method is presented in this dissertation. The analysis is based on identifying how operators identify and interpret signs depicted in the control room. Signs are, for example, information technological representations of process information, but they may also be the activities of other crew members. If the interpretation of signs has an identifiable global safety-related aspect, it may be concluded that the operating practice, even though situated, is also attuned to the general functions of the work which must always be maintained. A particular viewpoint in evaluating SU is provided by analysing user experiences (UX) which emerge in the complex work. For this purpose, a UX questionnaire was developed within the SU evaluation framework. The questionnaire is based on UX indicators which reflect experiences of appropriateness concerning the three tool functions (instrument, psychological, communicative). The importance of UX as a measure of germinating, not yet existing tool appropriateness, is presented in the dissertation. The contribution this dissertation makes is in the intersection of the research fields of human factors and ergonomics (HF/E) and usability engineering. The theoretical foundations of the research are in activity theory and cognitive systems engineering. The empirical work has been conducted by following the control room modernization efforts of Finnish nuclear power plants, during which evaluations have been carried out.

AB - The design of industrial control rooms assumes fulfilling the goals of production, safety, and human well-being. Control rooms and the user interfaces within them should enable the effective and efficient conduct of work in all the operating conditions which can be foreseen during specification and design. At the same time, the user interfaces should enable the control of the process in unprecedented and totally unexpected situations while simultaneously maintaining the safety of the process. During the design or modification of the control room, the potentiality of the emerging solution to fulfil the objectives is assessed by conducting empirical evaluations. This dissertation presents the development of an evaluation methodology which enables developing the control room towards meeting these objectives. Industrial process control constitutes a socio-technical system in which people and technologies have multiple and sometimes overlapping roles. In order to meet the demands of maintaining safety in all situations, the socio-technical system should have built-in capability of dealing with the unexpected. The control room, and the user interfaces within it, are an integral part of the socio-technical system. Thus, they have a role in construing and maintaining the safety of the system. The concept of systems usability (SU) is introduced in the dissertation to evaluate the systemic effects of control room solutions. SU is a human-centred quality attribute of user interfaces and control rooms attributed to technology, but the value in the use of the technology is evidenced in the success of the activity in which the technology is used. Thus, the research makes sense of the significance of the individual technological solutions in, and for, the entirety of an activity system. Systems usability means that a tool in an activity serves the functions of 1) an instrument, 2) a psychological tool, and 3) a communicative tool. The meaning of each function in the specific domain is contextually defined. Furthermore, the quality of the tool can be assessed utilising different perspectives on the usage activity: performance, way of acting, and user experience. By combining the functions of the tool and perspectives on activity, a systemic framework for developing contextual indicators for a good control room is construed. Utilising the concept of SU in the control room requires a model-based evaluation approach. This means that the general contextual work demands are considered in defining the reference of a successful process control activity. In addition, the scenarios utilised in the evaluation are modelled also taking the general work demands into account. A specific scenario modelling method, functional situation modelling (FSM), is presented in this dissertation. FSM combines a functional and a chronological view of the activity of an operating crew in a particular situation. By making explicit the connection between required operating actions and critical domain functions, the model lays the ground for analysing operating activity from the point of view of maintaining the critical functions. On aspect of control room evaluation is identifying whether the operating practices are attuned to maintaining safety in all situations. For this purpose, a data analysis method is presented in this dissertation. The analysis is based on identifying how operators identify and interpret signs depicted in the control room. Signs are, for example, information technological representations of process information, but they may also be the activities of other crew members. If the interpretation of signs has an identifiable global safety-related aspect, it may be concluded that the operating practice, even though situated, is also attuned to the general functions of the work which must always be maintained. A particular viewpoint in evaluating SU is provided by analysing user experiences (UX) which emerge in the complex work. For this purpose, a UX questionnaire was developed within the SU evaluation framework. The questionnaire is based on UX indicators which reflect experiences of appropriateness concerning the three tool functions (instrument, psychological, communicative). The importance of UX as a measure of germinating, not yet existing tool appropriateness, is presented in the dissertation. The contribution this dissertation makes is in the intersection of the research fields of human factors and ergonomics (HF/E) and usability engineering. The theoretical foundations of the research are in activity theory and cognitive systems engineering. The empirical work has been conducted by following the control room modernization efforts of Finnish nuclear power plants, during which evaluations have been carried out.

KW - control room evaluation

KW - human factors

KW - systems usability

KW - control room design

KW - nuclear power plant

M3 - Dissertation

SN - 978-951-38-8145-0

T3 - VTT Science

PB - VTT Technical Research Centre of Finland

CY - Espoo

ER -