Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.
- data pipeline
- big data
Ronkainen, J., & Iivari, A. (2015). Designing a Data Management Pipeline for Pervasive Sensor Communication Systems. Procedia Computer Science, 56, 183-188. https://doi.org/10.1016/j.procs.2015.07.193