Tsukada Lab. Computer Network Group

Cellular Connected UAV Anti-Interference Path Planning Based on PDS-DDPG and TOPEM

Quanxi Zhou, Yongjing Wang, Ruiyu Shen, Jin Nakazato, Manabu Tsukada, Zhenyu Guan

IEEE Journal on Miniaturization for Air and Space Systems, Nov 4, 2024

Summary in three lines

  • The study plans cellular-connected UAV routes to reduce both mission time and transmission outages.
  • It combines post-decision-state reinforcement learning with memory for transmission-outage probabilities.
  • The simulation reports faster training and fewer episodes to converge than basic DDPG.

Figures from the paper

Figure 8: Signaling in TOPEM-PDS-DDPG
Figure 8. Signaling in TOPEM-PDS-DDPGSource: Figure 8 from Zhou et al., IEEE Journal on Miniaturization for Air and Space Systems, 2024 (original artwork from the authors’ manuscript)The diagram identifies how communication measurements are passed into the framework for UAV route decisions.

Abstract

Due to the randomness of channel fading, communication devices, and malicious interference sources, unmanned aerial vehicles (UAVs) face a complex and ever-changing task scenario, which poses significant communication security challenges, such as transmission outages. Fortunately, these communication security challenges can be transformed into path planning problems that minimize the weighted sum of UAV mission time and transmission outage time. In order to design the complex communication environment faced by UAVs in actual scenarios, we propose a system model, including building distribution, communication channel, and antenna design in this paper. Besides, we introduce other UAVs with fixed flight paths and ground interference resources with random locations to ensure mission UAVs have better anti-interference ability. However, it is challenging for classical search algorithms and heuristic algorithms to cope with the complex path problems mentioned above. In this paper, we propose an improved deep deterministic policy gradient (DDPG) algorithm with better performance compared with basic DDPG and DDQN algorithms. Specifically, a post-decision state (PDS) mechanism has been introduced to accelerate the convergence rate and enhance the stability of the training process. In addition, a transmission outage probability experience memory (TOPEM) has been designed to quickly generate wireless communication quality maps and provide temporary experience for the post-decision process, resulting in better training results. Simulation experiments have proven that, compared to basic DDPG, the improved algorithm increases training speed by at least 50%, significantly improves convergence rate, and reduces the episode required for convergence to 20%. It can also help UAVs choose better paths than basic DDPG and DDQN algorithms.

Video

Related video: Trajectory Planning for UAV-Based Smart Farming Using Imitation-Based Triple Deep Q-Learning

Related research and software

How to cite

Q. Zhou, Y. Wang, R. Shen, J. Nakazato, M. Tsukada, and Z. Guan, “Cellular Connected UAV Anti-Interference Path Planning Based on PDS-DDPG and TOPEM,” IEEE Journal on Miniaturization for Air and Space Systems, 2024, doi: 10.1109/JMASS.2024.3490762.

BibTeX

@article{Zhou2024,
  title         = {Cellular Connected UAV Anti-Interference Path Planning Based on PDS-DDPG and TOPEM},
  author        = {Quanxi Zhou and Yongjing Wang and Ruiyu Shen and Jin Nakazato and Manabu Tsukada and Zhenyu Guan},
  doi           = {10.1109/JMASS.2024.3490762},
  issn          = {2576-3164},
  year          = {2024},
  date          = {2024-11-04},
  urldate       = {2024-11-04},
  journal       = {IEEE Journal on Miniaturization for Air and Space Systems},
  abstract      = {Due to the randomness of channel fading, communication devices, and malicious interference sources, unmanned aerial vehicles (UAVs) face a complex and ever-changing task scenario, which poses significant communication security challenges, such as transmission outages. Fortunately, these communication security challenges can be transformed into path planning problems that minimize the weighted sum of UAV mission time and transmission outage time. In order to design the complex communication environment faced by UAVs in actual scenarios, we propose a system model, including building distribution, communication channel, and antenna design in this paper. Besides, we introduce other UAVs with fixed flight paths and ground interference resources with random locations to ensure mission UAVs have better anti-interference ability. However, it is challenging for classical search algorithms and heuristic algorithms to cope with the complex path problems mentioned above. In this paper, we propose an improved deep deterministic policy gradient (DDPG) algorithm with better performance compared with basic DDPG and DDQN algorithms. Specifically, a post-decision state (PDS) mechanism has been introduced to accelerate the convergence rate and enhance the stability of the training process. In addition, a transmission outage probability experience memory (TOPEM) has been designed to quickly generate wireless communication quality maps and provide temporary experience for the post-decision process, resulting in better training results. Simulation experiments have proven that, compared to basic DDPG, the improved algorithm increases training speed by at least 50%, significantly improves convergence rate, and reduces the episode required for convergence to 20%. It can also help UAVs choose better paths than basic DDPG and DDQN algorithms.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {article}
}