Tsukada Lab. Computer Network Group

Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs

Quanxi Zhou, Wencan Mao, Jin Nakazato, Yusheng Ji, Manabu Tsukada

IEEE Transactions on Vehicular Technology, pp. 1–17, Sep 4, 2025

Summary in three lines

  • The study plans routes for cellular-connected UAVs in environments with uncertain interference.
  • It uses transmission-outage probability and a post-decision state to support multi-agent reinforcement learning.
  • Experiments evaluate the learned policies on mission time and communication outages.

Figures from the paper

Figure 2: Data-driven trajectory-planning methodology
Figure 2. Data-driven trajectory-planning methodologySource: Figure 2 from Zhou et al., IEEE Transactions on Vehicular Technology (2025) (authors’ figure from the manuscript)The workflow shows how environmental and communications models supply the learning problem for UAV route planning.

Abstract

Cellular-connected unmanned aerial vehicles (C-UAVs) will be an integral component of future wireless networks. Thanks to the mobility and maneuverability of UAVs, we can transform the interference management and route scheduling problems of C-UAVs into an anti-interference trajectory planning problem, aiming to jointly minimize the UAV mission time and transmission outage time. However, none of the existing methods have taken both the spatio-temporal uncertainty of interference sources and multi-UAV trajectory planning into consideration. To address this issue, we propose a novel method, referred to as uncertainty-aware multi-agent reinforcement learning (UA-MARL), for anti-interference trajectory planning of C-UAVs. In UA-MARL, a transmission outage probability (TOP) has been introduced to improve the robustness of the model. A transmission outage probability experience memory (TOPEM) has been designed to increase sample efficiency and reduce inference time. MARL algorithms integrated with an adaptive post-decision state (PDS) have been introduced to accelerate the convergence and stabilize the training. Experimental results show that UA-MARL outperforms baselines in average reward, convergence efficiency, and convergence stability. Furthermore, we find that higher residential density and wider considered area will lead to a decrease in training efficiency and stability.

Video

Related video: Trajectory Planning for UAV-Based Smart Farming Using Imitation-Based Triple Deep Q-Learning

Related research and software

How to cite

Q. Zhou, W. Mao, J. Nakazato, Y. Ji, and M. Tsukada, “Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs,” IEEE Transactions on Vehicular Technology, pp. 1–17, 2025, doi: 10.1109/TVT.2025.3606201.

BibTeX

@article{Zhou2025,
  title         = {Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs},
  author        = {Quanxi Zhou and Wencan Mao and Jin Nakazato and Yusheng Ji and Manabu Tsukada},
  doi           = {10.1109/TVT.2025.3606201},
  isbn          = {0018-9545},
  year          = {2025},
  date          = {2025-09-04},
  urldate       = {2025-09-09},
  journal       = {IEEE Transactions on Vehicular Technology},
  pages         = {1--17},
  abstract      = {Cellular-connected unmanned aerial vehicles (C-UAVs) will be an integral component of future wireless networks. Thanks to the mobility and maneuverability of UAVs, we can transform the interference management and route scheduling problems of C-UAVs into an anti-interference trajectory planning problem, aiming to jointly minimize the UAV mission time and transmission outage time. However, none of the existing methods have taken both the spatio-temporal uncertainty of interference sources and multi-UAV trajectory planning into consideration. To address this issue, we propose a novel method, referred to as uncertainty-aware multi-agent reinforcement learning (UA-MARL), for anti-interference trajectory planning of C-UAVs. In UA-MARL, a transmission outage probability (TOP) has been introduced to improve the robustness of the model. A transmission outage probability experience memory (TOPEM) has been designed to increase sample efficiency and reduce inference time. MARL algorithms integrated with an adaptive post-decision state (PDS) have been introduced to accelerate the convergence and stabilize the training. Experimental results show that UA-MARL outperforms baselines in average reward, convergence efficiency, and convergence stability. Furthermore, we find that higher residential density and wider considered area will lead to a decrease in training efficiency and stability.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {article}
}