Object detection in design diagrams with machine learning

Over the years companies have accumulated large amounts of legacy data. With modern data mining and machine learning techniques the data is increasingly valuable. Therefore being able to convert legacy data into a computer understandable form is important. In this work, we investigate how to convert schematic diagrams, such as process and instrumentation diagrams (P&I diagrams). We use modern machine learning based approaches, in particular, the Yolo neural network system, to detect high-level objects, e.g. pumps or valves, in diagrams which are scanned from paper archives or stored in pixel or vector form. Together with connection detection and OCR this is an essential step for the reuse of old planning data. Our results show that Yolo, as an instance of modern machine learning based object detection systems, works well with schematic diagrams. In our concept, we use a simulator to automatically generate labeled training material to the system. We then retrain a previously trained network to detect the components of our interest. Detection of large components is accurate but small components with sizes below 15% of page size are missed. However, this can be worked around by dividing a big diagram into a set of smaller subdiagrams with different scales, processing them separately, and combining the results.