New technique could help self-driving cars see their surroundings better

Thanks to a technique developed by researchers at NC State University, autonomous vehicles could one day travel better on the roads. The technique allows artificial intelligence programs to more accurately map three-dimensional spaces using two-dimensional images.

“Most autonomous vehicles use powerful AI programs called vision transformers to take 2D images from multiple cameras and create a representation of the 3D space around the vehicle,” says Tianfu Wu, associate professor of electrical engineering and computer science at NC State and corresponding author of a paper on the new technique. “However, while each of these AI programs takes a different approach, there is still much work to be done.”

Although these AI programs use different approaches, the new technique developed by Wu and his collaborators has the potential to significantly improve them all.

“Our technique, called Multi-View Attentive Contextualization (MvACon), is a plug-and-play add-on that can be used in conjunction with these existing vision transformer AIs to improve their ability to map 3D spaces,” explains Wu. The vision transformers don’t receive any additional data from their cameras, they are simply able to make better use of the data.”

The research team tested MvACon’s performance with three major vision transformers currently on the market, all of which rely on an array of six cameras to collect the 2D images they transform.

MvACon has significantly improved the performance of all three vision transformers.

“Performance was particularly improved when it came to locating objects, as well as the speed and orientation of those objects,” says Wu.

The research team presented the paper titled “Multi-View Attentive Contextualization for Multi-View 3D Object Detection” at this year’s IEEE/CVF Conference on Computer Vision and Pattern Recognition.