Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

Jussi Ronkainen, Antti Iivari

Research output: Contribution to journalArticleScientificpeer-review

1 Citation (Scopus)

Abstract

Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.
Original languageEnglish
Pages (from-to)183-188
JournalProcedia Computer Science
Volume56
DOIs
Publication statusPublished - 31 Jul 2015
MoE publication typeA1 Journal article-refereed

Fingerprint

Information management
Communication systems
Pipelines
Sensors
Monitoring
Processing
Costs

Keywords

  • data pipeline
  • sensors
  • iot
  • big data

Cite this

@article{6b12b3b0512b49df867cab642da77aac,
title = "Designing a Data Management Pipeline for Pervasive Sensor Communication Systems",
abstract = "Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.",
keywords = "data pipeline, sensors, iot, big data",
author = "Jussi Ronkainen and Antti Iivari",
year = "2015",
month = "7",
day = "31",
doi = "10.1016/j.procs.2015.07.193",
language = "English",
volume = "56",
pages = "183--188",
journal = "Procedia Computer Science",
issn = "1877-0509",
publisher = "Elsevier",

}

Designing a Data Management Pipeline for Pervasive Sensor Communication Systems. / Ronkainen, Jussi; Iivari, Antti.

In: Procedia Computer Science, Vol. 56, 31.07.2015, p. 183-188.

Research output: Contribution to journalArticleScientificpeer-review

TY - JOUR

T1 - Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

AU - Ronkainen, Jussi

AU - Iivari, Antti

PY - 2015/7/31

Y1 - 2015/7/31

N2 - Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.

AB - Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.

KW - data pipeline

KW - sensors

KW - iot

KW - big data

U2 - 10.1016/j.procs.2015.07.193

DO - 10.1016/j.procs.2015.07.193

M3 - Article

VL - 56

SP - 183

EP - 188

JO - Procedia Computer Science

JF - Procedia Computer Science

SN - 1877-0509

ER -