Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness

Hannu Karvonen; Eetu Heikkilä; Mikael Wahlström

doi:10.1007/978-3-030-49183-3_12

Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness

Hannu Karvonen (Corresponding author), Eetu Heikkilä, Mikael Wahlström

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

Abstract

Artificial intelligence (AI) is a key technology that is utilized in autonomous systems. However, using AI in such systems introduces also several safety challenges, many of which are related to complex human-AI interactions. In general, these challenges relate to the changing roles of the people who interact with the increasingly autonomous systems in various ways. In this paper, we consider a set of practical safety challenges related to applying advanced AI to autonomous machine systems and present solutions from the perspective of human factors. We apply the novel concept of AI awareness (AIA) to discuss the challenges in detail, and based on this, provide suggestions and guidelines for mitigating the AI safety risks with autonomous systems. In addition, we briefly consider the system design process to identify the actions that should be taken to ensure AIA throughout the different systems engineering design phases. The theoretical research presented in this paper aims to provide the first steps towards considering AIA in autonomous systems and understanding the human factors perspective viewpoint on AI safety for autonomous systems.

Original language	English
Title of host publication	Engineering Psychology and Cognitive Ergonomics. Cognition and Design
Subtitle of host publication	HCII 2020
Editors	Don Harris, Wen-Chin Li
Place of Publication	Cham
Publisher	Springer
Pages	147-160
Number of pages	14
ISBN (Electronic)	978-3-030-49183-3
ISBN (Print)	978-3-030-49182-6
DOIs	https://doi.org/10.1007/978-3-030-49183-3_12
Publication status	Published - 2020
MoE publication type	A4 Article in a conference publication
Event	17th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 - Copenhagen, Denmark Duration: 19 Jul 2020 → 24 Jul 2020

Publication series

Series	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12187
ISSN	0302-9743

Conference

Conference	17th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020
Country	Denmark
City	Copenhagen
Period	19/07/20 → 24/07/20

Keywords

Artificial intelligence
Autonomous systems
Human factors
Safety
Systems engineering

UN SDGs

This output contributes to the following Sustainable Development Goal(s)

Access to Document

10.1007/978-3-030-49183-3_12

Fingerprint Dive into the research topics of 'Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness'. Together they form a unique fingerprint.

Cite this

Karvonen, H., Heikkilä, E., & Wahlström, M. (2020). Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness. In D. Harris, & W-C. Li (Eds.), Engineering Psychology and Cognitive Ergonomics. Cognition and Design: HCII 2020 (pp. 147-160). Springer. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) Vol. 12187 https://doi.org/10.1007/978-3-030-49183-3_12

Karvonen, Hannu ; Heikkilä, Eetu ; Wahlström, Mikael. / Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness. Engineering Psychology and Cognitive Ergonomics. Cognition and Design: HCII 2020. editor / Don Harris ; Wen-Chin Li. Cham : Springer, 2020. pp. 147-160 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12187 ).

@inproceedings{9c9211d705a64bf182415480a4aeed5c,

title = "Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness",

abstract = "Artificial intelligence (AI) is a key technology that is utilized in autonomous systems. However, using AI in such systems introduces also several safety challenges, many of which are related to complex human-AI interactions. In general, these challenges relate to the changing roles of the people who interact with the increasingly autonomous systems in various ways. In this paper, we consider a set of practical safety challenges related to applying advanced AI to autonomous machine systems and present solutions from the perspective of human factors. We apply the novel concept of AI awareness (AIA) to discuss the challenges in detail, and based on this, provide suggestions and guidelines for mitigating the AI safety risks with autonomous systems. In addition, we briefly consider the system design process to identify the actions that should be taken to ensure AIA throughout the different systems engineering design phases. The theoretical research presented in this paper aims to provide the first steps towards considering AIA in autonomous systems and understanding the human factors perspective viewpoint on AI safety for autonomous systems.",

keywords = "Artificial intelligence, Autonomous systems, Human factors, Safety, Systems engineering",

author = "Hannu Karvonen and Eetu Heikkil{\"a} and Mikael Wahlstr{\"o}m",

year = "2020",

doi = "10.1007/978-3-030-49183-3_12",

language = "English",

isbn = "978-3-030-49182-6",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "147--160",

editor = "Don Harris and Wen-Chin Li",

booktitle = "Engineering Psychology and Cognitive Ergonomics. Cognition and Design",

address = "Germany",

note = "17th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020 ; Conference date: 19-07-2020 Through 24-07-2020",

}

Karvonen, H , Heikkilä, E & Wahlström, M 2020, Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness. in D Harris & W-C Li (eds), Engineering Psychology and Cognitive Ergonomics. Cognition and Design: HCII 2020. Springer, Cham, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12187 , pp. 147-160, 17th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020, Copenhagen, Denmark, 19/07/20. https://doi.org/10.1007/978-3-030-49183-3_12

Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness. / Karvonen, Hannu (Corresponding author); Heikkilä, Eetu ; Wahlström, Mikael.

Engineering Psychology and Cognitive Ergonomics. Cognition and Design: HCII 2020. ed. / Don Harris; Wen-Chin Li. Cham : Springer, 2020. p. 147-160 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12187 ).

Research output: Chapter in Book/Report/Conference proceeding › Conference article in proceedings › Scientific › peer-review

TY - GEN

T1 - Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness

AU - Karvonen, Hannu

AU - Heikkilä, Eetu

AU - Wahlström, Mikael

PY - 2020

Y1 - 2020

N2 - Artificial intelligence (AI) is a key technology that is utilized in autonomous systems. However, using AI in such systems introduces also several safety challenges, many of which are related to complex human-AI interactions. In general, these challenges relate to the changing roles of the people who interact with the increasingly autonomous systems in various ways. In this paper, we consider a set of practical safety challenges related to applying advanced AI to autonomous machine systems and present solutions from the perspective of human factors. We apply the novel concept of AI awareness (AIA) to discuss the challenges in detail, and based on this, provide suggestions and guidelines for mitigating the AI safety risks with autonomous systems. In addition, we briefly consider the system design process to identify the actions that should be taken to ensure AIA throughout the different systems engineering design phases. The theoretical research presented in this paper aims to provide the first steps towards considering AIA in autonomous systems and understanding the human factors perspective viewpoint on AI safety for autonomous systems.

AB - Artificial intelligence (AI) is a key technology that is utilized in autonomous systems. However, using AI in such systems introduces also several safety challenges, many of which are related to complex human-AI interactions. In general, these challenges relate to the changing roles of the people who interact with the increasingly autonomous systems in various ways. In this paper, we consider a set of practical safety challenges related to applying advanced AI to autonomous machine systems and present solutions from the perspective of human factors. We apply the novel concept of AI awareness (AIA) to discuss the challenges in detail, and based on this, provide suggestions and guidelines for mitigating the AI safety risks with autonomous systems. In addition, we briefly consider the system design process to identify the actions that should be taken to ensure AIA throughout the different systems engineering design phases. The theoretical research presented in this paper aims to provide the first steps towards considering AIA in autonomous systems and understanding the human factors perspective viewpoint on AI safety for autonomous systems.

KW - Artificial intelligence

KW - Autonomous systems

KW - Human factors

KW - Safety

KW - Systems engineering

UR - http://www.scopus.com/inward/record.url?scp=85088496628&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-49183-3_12

DO - 10.1007/978-3-030-49183-3_12

M3 - Conference article in proceedings

AN - SCOPUS:85088496628

SN - 978-3-030-49182-6

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 147

EP - 160

BT - Engineering Psychology and Cognitive Ergonomics. Cognition and Design

A2 - Harris, Don

A2 - Li, Wen-Chin

PB - Springer

CY - Cham

T2 - 17th International Conference on Engineering Psychology and Cognitive Ergonomics, EPCE 2020, held as part of the 22nd International Conference on Human-Computer Interaction, HCII 2020

Y2 - 19 July 2020 through 24 July 2020

ER -

Karvonen H , Heikkilä E , Wahlström M. Safety challenges of ai in autonomous systems design – solutions from human factors perspective emphasizing ai awareness. In Harris D, Li W-C, editors, Engineering Psychology and Cognitive Ergonomics. Cognition and Design: HCII 2020. Cham: Springer. 2020. p. 147-160. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), Vol. 12187 ). https://doi.org/10.1007/978-3-030-49183-3_12