Augmented/virtual reality can help extend the lifespan of critical infrastructure

Engineers at the University of Waterloo are turning to augmented/virtual reality (AR/VR) to better understand — and maintain — the physical reality of Canada’s critical infrastructure. Their research is published in the journal Automation in construction.

On-site inspectors are generally limited in what they can observe when inspecting bridges, roads, towers, pipelines and other structures because they cannot always see or detect all potential problems at within large infrastructures.

To help solve this problem, Waterloo Professor Chul Min Yeum and colleagues developed a cutting-edge system called Smart Infrastructure Metaverse that uses AR/VR to allow on-site and off-site inspectors to interact with each other. view both the real structure and a 3D scanned replica.

Not only does their system produce faster, more thorough, and more comprehensive results than traditional on-site visual exams, but it also provides broader context for issues within the entire facility.

Their innovative work meets an urgent need. Much of Canada’s vital infrastructure was built in the mid-20th century and is now near or past its useful life, a situation that poses serious risks to public safety. Yet replacing these structures would cost a staggering $264.7 billion, according to Canada’s Critical Public Infrastructure Survey. Yeum’s solution combines several advanced technologies so that problems can be resolved quickly and the lifespan of the infrastructure is extended.

“The Smart Infrastructure Metaverse aims to facilitate collaboration between on-site and off-site inspectors during structural inspections,” Yeum said. “We use AR/VR headsets to give them a split view so they can see exactly where they are and what they’re looking at during the inspection, no matter where they are.”

The research team included Yeum and Dr. Zaid Abbas Al-Sabbag, both from the Department of Civil and Environmental Engineering at Waterloo, as well as Dr. Sriram Narasimhan, from the Department of Civil and Environmental Engineering at the University of California, Los Angeles.

In a key experiment focused on a railway bridge in Kitchener, Ontario, Yeum and his colleagues created a three-dimensional model of the structure using 3D scanners and a panoramic camera. This model made it possible to accurately track the location and head position of on-site and off-site inspectors within the 3D map.

Then, an offsite engineer wearing a VR headset explored the 3D model of the bridge like someone would navigate a VR game. Meanwhile, on-site inspectors wearing AR headsets could, through a holographic display, see the actual bridge, the VR user and additional information on the digital map.

Since everyone participating in this examination was logged into the digital map, the off-site inspector saw the exact location of the on-site users as well as the areas of the structure they were inspecting. This meant that the remote inspector could guarantee the accuracy of the data collected. To support human inspectors, the research team used artificial intelligence to analyze images sent from on-site inspectors’ AR headsets to further identify any structural damage.