Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

Jussi Ronkainen, Antti Iivari

    Research output: Contribution to journalArticleScientificpeer-review

    1 Citation (Scopus)

    Abstract

    Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.
    Original languageEnglish
    Pages (from-to)183-188
    JournalProcedia Computer Science
    Volume56
    DOIs
    Publication statusPublished - 31 Jul 2015
    MoE publication typeA1 Journal article-refereed

    Fingerprint

    Information management
    Communication systems
    Pipelines
    Sensors
    Monitoring
    Processing
    Costs

    Keywords

    • data pipeline
    • sensors
    • iot
    • big data

    Cite this

    @article{6b12b3b0512b49df867cab642da77aac,
    title = "Designing a Data Management Pipeline for Pervasive Sensor Communication Systems",
    abstract = "Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.",
    keywords = "data pipeline, sensors, iot, big data",
    author = "Jussi Ronkainen and Antti Iivari",
    year = "2015",
    month = "7",
    day = "31",
    doi = "10.1016/j.procs.2015.07.193",
    language = "English",
    volume = "56",
    pages = "183--188",
    journal = "Procedia Computer Science",
    issn = "1877-0509",
    publisher = "Elsevier",

    }

    Designing a Data Management Pipeline for Pervasive Sensor Communication Systems. / Ronkainen, Jussi; Iivari, Antti.

    In: Procedia Computer Science, Vol. 56, 31.07.2015, p. 183-188.

    Research output: Contribution to journalArticleScientificpeer-review

    TY - JOUR

    T1 - Designing a Data Management Pipeline for Pervasive Sensor Communication Systems

    AU - Ronkainen, Jussi

    AU - Iivari, Antti

    PY - 2015/7/31

    Y1 - 2015/7/31

    N2 - Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.

    AB - Pervasive sensor systems offer unbounded possibilities for monitoring and tracking objects, machines, and spaces. To maximize the benefit from a sensor system, sensor data requires efficient preprocessing and analysis. Big data techniques make distributed processing of huge amounts of data fast and cost-effective, making them a practical necessity for sensor data. However, the real-time requirements and the sheer velocity and volume of data from large sensor systems require a dedicated approach to designing the data processing pipeline. This paper discusses viewpoints and requirements for designing a sensor data pipeline, with specific focus on data input, live preprocessing, and storage.

    KW - data pipeline

    KW - sensors

    KW - iot

    KW - big data

    U2 - 10.1016/j.procs.2015.07.193

    DO - 10.1016/j.procs.2015.07.193

    M3 - Article

    VL - 56

    SP - 183

    EP - 188

    JO - Procedia Computer Science

    JF - Procedia Computer Science

    SN - 1877-0509

    ER -