Fusion of Sound Source Localization and Face Detection for Supporting Human Behavior Analysis

This paper describes a demonstrated concept implementation that combines sound source localization and face detection from video stream for supporting human behavior analysis. System monitors space containing multiple persons using microphone array and video camera. The aim is to detect which person in the scene is producing the sound that is received by the microphones. For this task the microphone array localizes the sound in the environment. Simultaneously face detection is performed to the video signal produced by the monitoring video camera. If face is detected from the bearing of the sound the system may decide that the sound is produced by the person who's face is detected. Preliminary results indicate that the fusion may give useful information for human behavior analysis for space containing multiple persons.