Tsukada Lab. Computer Network Group

Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs

Quanxi Zhou, Wencan Mao, Jin Nakazato, Yusheng Ji, Manabu Tsukada

IEEE Transactions on Vehicular Technology, pp. 1–17, 2025年9月4日

3行要約

  • 干渉源の状態が不確かな環境で、セルラー網につながるUAVの航路を計画する。
  • 伝送途絶確率と意思決定直後の状態を複数エージェントの強化学習に取り込む。
  • 学習した方策を、任務時間と通信途絶時間の観点から評価した。

論文の図

図 2: データ駆動型の軌道計画手法
図 2. データ駆動型の軌道計画手法出典: Zhou et al., IEEE Transactions on Vehicular Technology (2025)(著者原稿の図版) の図 2環境と通信のモデルを、UAVの航路計画の学習に取り込む手順が分かる。

アブストラクト(原文)

Cellular-connected unmanned aerial vehicles (C-UAVs) will be an integral component of future wireless networks. Thanks to the mobility and maneuverability of UAVs, we can transform the interference management and route scheduling problems of C-UAVs into an anti-interference trajectory planning problem, aiming to jointly minimize the UAV mission time and transmission outage time. However, none of the existing methods have taken both the spatio-temporal uncertainty of interference sources and multi-UAV trajectory planning into consideration. To address this issue, we propose a novel method, referred to as uncertainty-aware multi-agent reinforcement learning (UA-MARL), for anti-interference trajectory planning of C-UAVs. In UA-MARL, a transmission outage probability (TOP) has been introduced to improve the robustness of the model. A transmission outage probability experience memory (TOPEM) has been designed to increase sample efficiency and reduce inference time. MARL algorithms integrated with an adaptive post-decision state (PDS) have been introduced to accelerate the convergence and stabilize the training. Experimental results show that UA-MARL outperforms baselines in average reward, convergence efficiency, and convergence stability. Furthermore, we find that higher residential density and wider considered area will lead to a decrease in training efficiency and stability.

動画

関連する動画: Trajectory Planning for UAV-Based Smart Farming Using Imitation-Based Triple Deep Q-Learning

関連する研究・ソフトウェア

引用方法

Q. Zhou, W. Mao, J. Nakazato, Y. Ji, and M. Tsukada, “Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs,” IEEE Transactions on Vehicular Technology, pp. 1–17, 2025, doi: 10.1109/TVT.2025.3606201.

BibTeX

@article{Zhou2025,
  title         = {Uncertainty-Aware Multi-Agent Reinforcement Learning for Anti-Interference Trajectory Planning of Cellular-Connected UAVs},
  author        = {Quanxi Zhou and Wencan Mao and Jin Nakazato and Yusheng Ji and Manabu Tsukada},
  doi           = {10.1109/TVT.2025.3606201},
  isbn          = {0018-9545},
  year          = {2025},
  date          = {2025-09-04},
  urldate       = {2025-09-09},
  journal       = {IEEE Transactions on Vehicular Technology},
  pages         = {1--17},
  abstract      = {Cellular-connected unmanned aerial vehicles (C-UAVs) will be an integral component of future wireless networks. Thanks to the mobility and maneuverability of UAVs, we can transform the interference management and route scheduling problems of C-UAVs into an anti-interference trajectory planning problem, aiming to jointly minimize the UAV mission time and transmission outage time. However, none of the existing methods have taken both the spatio-temporal uncertainty of interference sources and multi-UAV trajectory planning into consideration. To address this issue, we propose a novel method, referred to as uncertainty-aware multi-agent reinforcement learning (UA-MARL), for anti-interference trajectory planning of C-UAVs. In UA-MARL, a transmission outage probability (TOP) has been introduced to improve the robustness of the model. A transmission outage probability experience memory (TOPEM) has been designed to increase sample efficiency and reduce inference time. MARL algorithms integrated with an adaptive post-decision state (PDS) have been introduced to accelerate the convergence and stabilize the training. Experimental results show that UA-MARL outperforms baselines in average reward, convergence efficiency, and convergence stability. Furthermore, we find that higher residential density and wider considered area will lead to a decrease in training efficiency and stability.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {article}
}