An Automatic Registration System Based on Augmented Reality to Enhance Civil Infrastructure Inspections

Manual geometric and semantic alignment of inspection data with existing digital models (field-to-model data registration) and on-site access to relevant information (model-to-field data registration) represent cumbersome procedures that cause significant loss of information and fragmentation, hindering the efficiency of civil infrastructure inspections. To address the bidirectional registration challenge, this study introduces a high-accuracy automatic registration method and system based on Augmented Reality (AR) that streamlines data exchange between the field and a knowledge graph-based Digital Twin (DT) platform for infrastructure management, and vice versa. A centimeter-level 6-DoF pose estimation of the AR device in large-scale, open unprepared environments is achieved by implementing a hybrid approach based on Real-Time Kinematic and Visual Inertial Odometry to cope with urban-canyon scenarios. For this purpose, a low-cost and non-invasive RTK receiver was prototyped and firmly attached to an AR device (i.e., Microsoft HoloLens 2). Multiple filters and latency compensation techniques were implemented to enhance registration accuracy. The system was tested in a real-world scenario involving the inspection of a highway viaduct. Throughout the use case inspection, the system seamlessly and automatically provided field operators with on-field access to existing DT information (i.e., open BIM models) such as georeferenced holograms and facilitated the enrichment of the asset’s DT through the automatic registration of inspection data (i.e., images) with the open BIM models included in the DT. This study contributes to DT-based civil infrastructure management by establishing a bidirectional and seamless integration between virtual and physical entities.