Object detection in design diagrams with machine learning

Jukka K. Nurminen, Kari Rainio (Corresponding author), Jukka Pekka Numminen, Timo Syrjänen, Niklas Paganus, Karri Honkoila

Research output: Chapter in Book/Report/Conference proceedingChapter or book articleScientificpeer-review

Abstract

Over the years companies have accumulated large amounts of legacy data. With modern data mining and machine learning techniques the data is increasingly valuable. Therefore being able to convert legacy data into a computer understandable form is important. In this work, we investigate how to convert schematic diagrams, such as process and instrumentation diagrams (P&I diagrams). We use modern machine learning based approaches, in particular, the Yolo neural network system, to detect high-level objects, e.g. pumps or valves, in diagrams which are scanned from paper archives or stored in pixel or vector form. Together with connection detection and OCR this is an essential step for the reuse of old planning data. Our results show that Yolo, as an instance of modern machine learning based object detection systems, works well with schematic diagrams. In our concept, we use a simulator to automatically generate labeled training material to the system. We then retrain a previously trained network to detect the components of our interest. Detection of large components is accurate but small components with sizes below 15% of page size are missed. However, this can be worked around by dividing a big diagram into a set of smaller subdiagrams with different scales, processing them separately, and combining the results.

Original languageEnglish
Title of host publicationCORES 2019
Subtitle of host publicationProgress in Computer Recognition Systems
PublisherSpringer
Pages27-36
Number of pages10
ISBN (Electronic)978-3-030-19738-4
ISBN (Print)978-3-030-19737-7
DOIs
Publication statusPublished - 1 Jan 2020
MoE publication typeA3 Part of a book or another research book
EventInternational Conference on Computer Recognition Systems, CORES 2019 - Polanica-Zdrój, Poland
Duration: 20 May 202022 May 2020

Publication series

SeriesAdvances in Intelligent Systems and Computing
Volume977
ISSN2194-5357

Conference

ConferenceInternational Conference on Computer Recognition Systems, CORES 2019
Abbreviated titleCORES 2019
CountryPoland
CityPolanica-Zdrój
Period20/05/2022/05/20

Fingerprint

Learning systems
Schematic diagrams
Optical character recognition
Data mining
Simulators
Pixels
Pumps
Neural networks
Planning
Processing
Object detection
Industry

Keywords

  • Legacy data
  • Machine learning
  • Object detection
  • Schematic diagrams

Cite this

Nurminen, J. K., Rainio, K., Numminen, J. P., Syrjänen, T., Paganus, N., & Honkoila, K. (2020). Object detection in design diagrams with machine learning. In CORES 2019: Progress in Computer Recognition Systems (pp. 27-36). Springer. Advances in Intelligent Systems and Computing, Vol.. 977 https://doi.org/10.1007/978-3-030-19738-4_4
Nurminen, Jukka K. ; Rainio, Kari ; Numminen, Jukka Pekka ; Syrjänen, Timo ; Paganus, Niklas ; Honkoila, Karri. / Object detection in design diagrams with machine learning. CORES 2019: Progress in Computer Recognition Systems. Springer, 2020. pp. 27-36 (Advances in Intelligent Systems and Computing, Vol. 977).
@inbook{58f4d0384d1b49379600fb370835f87d,
title = "Object detection in design diagrams with machine learning",
abstract = "Over the years companies have accumulated large amounts of legacy data. With modern data mining and machine learning techniques the data is increasingly valuable. Therefore being able to convert legacy data into a computer understandable form is important. In this work, we investigate how to convert schematic diagrams, such as process and instrumentation diagrams (P&I diagrams). We use modern machine learning based approaches, in particular, the Yolo neural network system, to detect high-level objects, e.g. pumps or valves, in diagrams which are scanned from paper archives or stored in pixel or vector form. Together with connection detection and OCR this is an essential step for the reuse of old planning data. Our results show that Yolo, as an instance of modern machine learning based object detection systems, works well with schematic diagrams. In our concept, we use a simulator to automatically generate labeled training material to the system. We then retrain a previously trained network to detect the components of our interest. Detection of large components is accurate but small components with sizes below 15{\%} of page size are missed. However, this can be worked around by dividing a big diagram into a set of smaller subdiagrams with different scales, processing them separately, and combining the results.",
keywords = "Legacy data, Machine learning, Object detection, Schematic diagrams",
author = "Nurminen, {Jukka K.} and Kari Rainio and Numminen, {Jukka Pekka} and Timo Syrj{\"a}nen and Niklas Paganus and Karri Honkoila",
year = "2020",
month = "1",
day = "1",
doi = "10.1007/978-3-030-19738-4_4",
language = "English",
isbn = "978-3-030-19737-7",
series = "Advances in Intelligent Systems and Computing",
publisher = "Springer",
pages = "27--36",
booktitle = "CORES 2019",
address = "Germany",

}

Nurminen, JK, Rainio, K, Numminen, JP, Syrjänen, T, Paganus, N & Honkoila, K 2020, Object detection in design diagrams with machine learning. in CORES 2019: Progress in Computer Recognition Systems. Springer, Advances in Intelligent Systems and Computing, vol. 977, pp. 27-36, International Conference on Computer Recognition Systems, CORES 2019, Polanica-Zdrój, Poland, 20/05/20. https://doi.org/10.1007/978-3-030-19738-4_4

Object detection in design diagrams with machine learning. / Nurminen, Jukka K.; Rainio, Kari (Corresponding author); Numminen, Jukka Pekka; Syrjänen, Timo; Paganus, Niklas; Honkoila, Karri.

CORES 2019: Progress in Computer Recognition Systems. Springer, 2020. p. 27-36 (Advances in Intelligent Systems and Computing, Vol. 977).

Research output: Chapter in Book/Report/Conference proceedingChapter or book articleScientificpeer-review

TY - CHAP

T1 - Object detection in design diagrams with machine learning

AU - Nurminen, Jukka K.

AU - Rainio, Kari

AU - Numminen, Jukka Pekka

AU - Syrjänen, Timo

AU - Paganus, Niklas

AU - Honkoila, Karri

PY - 2020/1/1

Y1 - 2020/1/1

N2 - Over the years companies have accumulated large amounts of legacy data. With modern data mining and machine learning techniques the data is increasingly valuable. Therefore being able to convert legacy data into a computer understandable form is important. In this work, we investigate how to convert schematic diagrams, such as process and instrumentation diagrams (P&I diagrams). We use modern machine learning based approaches, in particular, the Yolo neural network system, to detect high-level objects, e.g. pumps or valves, in diagrams which are scanned from paper archives or stored in pixel or vector form. Together with connection detection and OCR this is an essential step for the reuse of old planning data. Our results show that Yolo, as an instance of modern machine learning based object detection systems, works well with schematic diagrams. In our concept, we use a simulator to automatically generate labeled training material to the system. We then retrain a previously trained network to detect the components of our interest. Detection of large components is accurate but small components with sizes below 15% of page size are missed. However, this can be worked around by dividing a big diagram into a set of smaller subdiagrams with different scales, processing them separately, and combining the results.

AB - Over the years companies have accumulated large amounts of legacy data. With modern data mining and machine learning techniques the data is increasingly valuable. Therefore being able to convert legacy data into a computer understandable form is important. In this work, we investigate how to convert schematic diagrams, such as process and instrumentation diagrams (P&I diagrams). We use modern machine learning based approaches, in particular, the Yolo neural network system, to detect high-level objects, e.g. pumps or valves, in diagrams which are scanned from paper archives or stored in pixel or vector form. Together with connection detection and OCR this is an essential step for the reuse of old planning data. Our results show that Yolo, as an instance of modern machine learning based object detection systems, works well with schematic diagrams. In our concept, we use a simulator to automatically generate labeled training material to the system. We then retrain a previously trained network to detect the components of our interest. Detection of large components is accurate but small components with sizes below 15% of page size are missed. However, this can be worked around by dividing a big diagram into a set of smaller subdiagrams with different scales, processing them separately, and combining the results.

KW - Legacy data

KW - Machine learning

KW - Object detection

KW - Schematic diagrams

UR - http://www.scopus.com/inward/record.url?scp=85065814159&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-19738-4_4

DO - 10.1007/978-3-030-19738-4_4

M3 - Chapter or book article

AN - SCOPUS:85065814159

SN - 978-3-030-19737-7

T3 - Advances in Intelligent Systems and Computing

SP - 27

EP - 36

BT - CORES 2019

PB - Springer

ER -

Nurminen JK, Rainio K, Numminen JP, Syrjänen T, Paganus N, Honkoila K. Object detection in design diagrams with machine learning. In CORES 2019: Progress in Computer Recognition Systems. Springer. 2020. p. 27-36. (Advances in Intelligent Systems and Computing, Vol. 977). https://doi.org/10.1007/978-3-030-19738-4_4