LiDAR-based system allows team of unmanned aerial vehicles to quickly reconstruct environments

Unmanned aerial vehicles (UAVs), commonly known as drones, have proven to be very effective systems for monitoring and exploring environments. These autonomous flying robots could also be used to create detailed maps and three-dimensional (3D) visualizations of real-world environments.

Researchers from Sun Yat-Sen University and the Hong Kong University of Science and Technology recently introduced SOAR, a system that allows a team of drones to quickly and autonomously reconstruct environments by exploring and photographing them simultaneously. This system, presented in an article published on the arXiv preprint server and assembly that will be presented at the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024, could have many applications, ranging from urban planning to the design of video game environments.

“Our paper arises from the growing need for efficient, high-quality 3D reconstruction using drones,” Mingjie Zhang, co-author of the paper, told Tech Xplore.

“We observed that existing methods often fall into two categories: model-based approaches, which can be time-consuming and expensive due to their reliance on prior information, and model-free methods, which explore and reconstruct simultaneously but may be limited by local planning constraints. Our goal was to bridge this gap by developing a system capable of leveraging the strengths of both approaches.

The main goal of the recent study by Zhang and colleagues was to create a heterogeneous multi-UAV system capable of simultaneously exploring environments and collecting photographs, thereby collecting data that can be used to reconstruct environments. To do this, they first set out to develop an incremental viewpoint generation technique that adapts to scene information acquired over time.

Additionally, the team planned to develop a tasking strategy that would maximize the efficiency of the multi-UAV team, ensuring that it systematically collected the data needed to reconstruct the environments. Finally, the team performed a series of simulations to evaluate the effectiveness of the proposed system.

“SOAR is a LiDAR-Visual heterogeneous multi-UAV system designed for rapid autonomous 3D reconstruction,” Zhang explained. “It employs a team of drones: an explorer equipped with LiDAR for rapid exploration of scenes and several photographers equipped with cameras to capture detailed images.”

To create 3D reconstructions, the system proposed by the team involves several steps. First, a drone they call the “Explorer” effectively navigates and maps an environment using a strategy based on surface boundaries.

As this drone maps the environment, the team’s system gradually generates viewpoints that would collectively allow the surfaces of the bounded environment to be completely covered. Other drones, called photographers, will then visit these sites and collect visual data.

“Viewpoints are grouped and assigned to photographers using the Consistent-MDMTSP method, balancing the workload and maintaining task consistency,” Zhang said. “Each photographer plans an optimal path for capturing images from assigned viewpoints. The collected images and their corresponding poses are then used to generate a textured 3D model.”

A unique feature of SOAR is that it allows data collection by both LiDAR and visual sensors. This ensures efficient exploration of environments and production of high-quality reconstructions.

“Our system adapts to dynamically changing scene information, ensuring optimal coverage with minimal viewpoints,” Zhang said. “By systematically assigning tasks to drones, it also improves scanning efficiency and reduces unnecessary detours for photographers.”

Zhang and his colleagues evaluated the proposed system in a series of simulations. Their results were very promising, as SOAR was found to outperform other state-of-the-art methods for environmental reconstruction.

“A key achievement of our study is the introduction of a new framework for autonomous and rapid aerial reconstruction,” Zhang said. “At the heart of this framework is the development of several key algorithms that use an incremental design, striking a crucial balance between real-time scheduling capabilities and overall efficiency, which is essential for online reconstruction tasks and dynamic. »

In the future, SOAR could be used to solve a wide range of real-world problems requiring rapid and accurate reconstruction of 3D environments. For example, it could be used to create detailed 3D models of cities and infrastructure or help historians preserve a country’s cultural heritage, helping them reconstruct historical sites and artifacts.

“SOAR could also be used for disaster response and assessment,” Zhang said. “Specifically, it could allow responders to quickly assess damage caused by natural disasters and plan rescue and recovery efforts.”

The team’s system could further aid in the inspection of infrastructure and construction sites, allowing workers to clearly map these locations. Finally, it could be used to create 3D models of video game environments inspired by real cities and natural landscapes.

“We are excited about the potential for future research in this area,” Zhang said. “Our plans include bridging the gap between simulation and the real world: we aim to address the challenges associated with transitioning SOAR from simulation to real-world environments. This will involve resolving issues such as localization errors and communication interruptions that can occur during real-world deployments. “.

As part of their next studies, the researchers plan to develop new task assignment strategies that could further improve the coordination between different drones and the speed at which they map environments. Finally, they plan to add scene prediction and information processing modules to their system, as this could allow it to anticipate the structure of a given environment, thus further speeding up the reconstruction process.

“We will also explore the implementation of active reconstruction techniques, where the system receives real-time feedback during the reconstruction process,” Zhang added.

“This will allow SOAR to adapt its planning on the fly and achieve even better results. Additionally, we will explore integrating factors such as camera angle and image quality directly into the planning process, which will ensure that captured images are optimized to generate high-quality 3D reconstructions. These research directions represent exciting opportunities to advance the capabilities of SOAR and push the boundaries of autonomous 3D reconstruction using. drones.

© 2024 Science X Network