A 360-degree heads-up display could warn drivers of road obstacles in real time

Researchers have developed an augmented reality head-up display that could improve road safety by displaying potential hazards as high-resolution three-dimensional holograms directly in the driver’s field of vision in real time.

Current head-up display systems are limited to two-dimensional projections on a vehicle’s windshield, but researchers from the Universities of Cambridge, Oxford and University College London (UCL) have developed a system using a 3D laser scanner and LiDAR data to create a complete display. 3D representation of the streets of London.

The system they developed can effectively “see” through objects to project holographic representations of road obstacles hidden from the driver’s field of vision, aligned with the real object in terms of size and distance. For example, a road sign obscured by a large truck would appear as a 3D hologram so the driver knows exactly where the sign is and what information it displays.

3D holographic projection technology keeps the driver focused on the road rather than the windshield, and could improve road safety by projecting road obstacles and potential hazards in real time from any angle. The results are reported in the journal Advanced optical materials.

Every day, around 16,000 people die in road accidents caused by human error. Technology could be used to reduce this number and improve road safety, in part by providing information to drivers about potential dangers. Currently this is mainly done using head-up displays, which can provide information such as current speed or driving directions.

“The idea behind a head-up display is that it allows the driver to keep their eyes open, because even a fraction of a second without looking at the road is enough for an accident to occur,” said Jana Skirnewskaja of the Department of Transportation. of Cambridge Engineering, the first author of the study. “However, because they are two-dimensional images, projected onto a small area of ​​the (windshield), the driver can look at the image and not actually look at the road ahead.”

For several years, Skirnewskaja and her colleagues have been working to develop alternatives to heads-up displays (HUDs) that could improve road safety by providing more accurate information to drivers while keeping their eyes on the road.

“We want to project information anywhere in the driver’s field of vision, but in a way that isn’t overwhelming or distracting,” Skirnewskaja said. “We do not wish to provide any information that is not directly related to the driving task at hand.”

The team developed an augmented reality holographic point cloud video projection system to display objects aligned with real objects in size and distance within the driver’s field of vision. The system combines data from a 3D holographic setup with LiDAR (light detection and ranging) data. LiDAR uses a pulsed light source to illuminate an object and the reflected light pulses are then measured to calculate the distance between the object and the light source.

Researchers tested the system by scanning Malet Street on the UCL campus in central London. Information from the LiDAR point cloud was transformed into layered 3D holograms, comprising up to 400,000 data points. The concept of projecting a 360° obstacle assessment for drivers was born from careful data processing, ensuring clear visibility of the depth of each object.

The researchers sped up the scanning process so that the holograms were generated and projected in real time. Importantly, analytics can provide dynamic insights as busy streets change from moment to moment.

“The data we collected can be shared and stored in the cloud, so that all passing drivers have access to it. It’s like a more sophisticated version of the navigation apps we use every day to provide traffic information in real time,” Skirnewskaja said. “This way, the system is dynamic and can adapt to changing conditions, as hazards or obstacles move in or out of the street.”

Although collecting data from diverse locations improves accuracy, the researchers say the unique contribution of their study is the ability to gain a 360-degree view by judiciously choosing data points from single analyzes of specific objects, such as trucks or buildings, allowing a comprehensive assessment of road conditions. dangers.

“We can scan up to 400,000 data points for a single object, but that obviously requires a lot of data and makes it more difficult to scan, extract and project data onto that object in real time,” Skirnewskaja said. . “With just 100 data points, we can know what the object is and how big it is. We need to get just enough information for the driver to know what’s around it.”

Earlier this year, Skirnewskaja and colleagues performed a virtual demonstration using virtual reality headsets loaded with the system’s LiDAR data at the Science Museum in London. User feedback during the sessions helped the researchers improve the system to make the design more inclusive and user-friendly. For example, they refined the system to reduce eye fatigue and took visual impairments into account.

“We want a system that is accessible and inclusive, so that end users are comfortable with it,” Skirnewskaja said. “If the system is a distraction, then it doesn’t work. We want something that is helpful to drivers and improves the safety of all road users, including pedestrians and cyclists.”

Researchers are currently working with Google to develop the technology so it can be tested on real cars. They hope to carry out road tests, on public or private roads, in 2024.