Video-based fusion of multiple detectors to counter terrorism

Luigi de Dominicis, Henri Bouma, Sirra Toivonen, Cristiano Stifini, Maria Luisa Villani, Antonio de Nicola, Arthur van Rooijen, Jan Baan, Johannes Peltola, Arttu Lämsä, Pauli Räsänen

Research output: Chapter in Book/Report/Conference proceedingConference article in proceedingsScientificpeer-review

Abstract

Terrorism is an international security challenge. The early detection of threats (e.g., explosives or firearms) could provide a valuable contribution to the ability to prevent, protect and respond to terrorism. This paper presents a system for the management of a plurality of sensors to improve the threat-detection capabilities without disrupting the flow of passengers. The system improves the prevention capabilities of soft targets (such as airports, undergrounds and railway stations) with a high number of daily commuters. The system architecture consists of three main components. The first component is 2D video tracking and re-identification (Re-ID), which allows the labelling and tracking of commuters in a small area. Thereby, it supports the fusion of sensors at different locations. The Re-ID has a smart training strategy with anonymized snippets to increase flexibility for new environments. The second component is 3D video tracking with a stereo camera, which gives a more accurate location measurement than 2D video. Location prediction is used to compensate for latency in the control of active elements in the threat detection sensor. Recurrent neural networks for location prediction were trained by using real 3D tracking data from a railway station. The performance is evaluated with a ground-truth based on Ultra-Wide Band (UWB) radio positioning and a coordinate conversion method was created to compensate for identified inaccuracies. The third component is Command & Control (C&C), which consists of three submodules: message broker, data-fusion and security client. The message broker is a publish-subscribe middleware layer to enable flexible integration of the various sensors and components. The data-fusion combines outputs of multiple sensors. In case of a suspect person, the security client triggers an alarm and a comprehensive report is sent to the security guards.
Original languageEnglish
Title of host publicationCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies V
EditorsH. Bouma, R. Prabhu, R. J. Stokes, Y. Yitzhaky
PublisherInternational Society for Optics and Photonics SPIE
ISBN (Electronic)978-1-5106-4582-0
DOIs
Publication statusPublished - 14 Oct 2021
MoE publication typeA4 Article in a conference publication
EventCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies V 2021: Online - Virtual, Spain
Duration: 13 Sep 202117 Sep 2021

Publication series

SeriesProceedings of SPIE - The International Society for Optical Engineering
Volume11869
ISSN0277-786X

Conference

ConferenceCounterterrorism, Crime Fighting, Forensics, and Surveillance Technologies V 2021
Country/TerritorySpain
Period13/09/2117/09/21

Keywords

  • Command and control
  • Counter-terrorism
  • Explosives
  • Firearms
  • Fusion
  • Re-identification
  • Surveillance
  • Threat detection

Fingerprint

Dive into the research topics of 'Video-based fusion of multiple detectors to counter terrorism'. Together they form a unique fingerprint.

Cite this