Autonomous Operation of UAS in Complex Environments Using Data- and Vision-Based Deep Reinforcement Learning and Validation by AirSim and Flight Test

Abstract

Real-time autonomous path planning is a critical capability in unmanned aerial vehicle (UAV) operations, especially in complex environments that involve both static and dynamic obstacles. This study proposes a dual-layer unmanned mobility framework that integrates dynamic obstacle avoidance based on data communication with static obstacle avoidance based on vision. The data communication-based layer utilizes the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm to process real-time position and velocity data from the Ground Control System (GCS), thereby reducing collision risks and enabling safe route planning in multi-UAV operations. The vision-based layer utilizes a depth camera to detect and avoid previously unknown or unpredictable obstacles, enabling real-time trajectory adjustments without relying on pre-mapped data. The dual-layer UAS operation system underwent a rigorous multi-phase validation process. Initially, the system was validated through Software-in-the-Loop (SITL) simulations conducted using Microsoft’s AirSim platform, which provided a high-fidelity environment for realistic training and performance assessment. SITL enabled effective testing of decision-making algorithms in complex environments containing both static and dynamic obstacles. To further enhance the reliability of the proposed system and assess its performance under hardware constraints, Hardware-in-the-Loop (HITL) testing was subsequently implemented as an intermediate verification step. The HITL configuration enabled real-time assessment of attitude and angular rate tracking through flight controllers, ensuring system stability and precise control before actual flight testing. Finally, actual flight tests were conducted at the drone test field of the Gwangju Institute of Science and Technology (GIST). Consistent results observed across SITL, HITL, and flight tests validated the system’s practicality, robustness, and applicability to real-world UAV operations in complex obstacle environments. This dual-layer hybrid framework demonstrates the effective integration of data communication and vision-based strategies, enabling reliable and efficient UAV navigation in complex environments. The findings support the potential of this system for advanced UAV applications in urban logistics, military missions, and disaster response operations. © 2025 Elsevier B.V., All rights reserved.