Tsukada Lab. Computer Network Group

MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing

Quanxi Zhou, Wencan Mao, Yunfei Chen, Junru Li, Jiachen Gong, Kensuke Fukuda, Manabu Tsukada

21st Asian Internet Engineering Conference (AINTEC 2026), Taipei, Taiwan, 2026年12月1日

3行要約

  • MPDS-SACは都市の地上センサからデータを集めるUAVの軌道を学習する。
  • 行動の予測可能な影響を2段先まで計算し、環境との追加相互作用なしでクリティックの学習に使う。
  • UAV 8機の都市シミュレーションで情報鮮度が向上し、SACやTD3より早く収束した。

論文の図

図 1: 都市の空地協調センシングシステム
図 1. 都市の空地協調センシングシステム出典: Zhou et al., 21st Asian Internet Engineering Conference (AINTEC 2026) (2026)(著者原稿の図版) の図 1地上センサからUAVが情報を集め、基地局と通信する構成を示す。
図 3: 各方策の学習報酬
図 3. 各方策の学習報酬出典: Zhou et al., 21st Asian Internet Engineering Conference (AINTEC 2026) (2026)(著者原稿の図版) の図 3シミュレーションの学習曲線から、MPDS-SACが比較手法より早く高い報酬に達することを確認できる。

アブストラクト(原文)

Air-ground collaborative sensing, which utilizes unmanned aerial vehicles (UAVs) to collect data from ground wireless sensors, facilitates smart-city monitoring benefiting from mobility, scalability, and fast responses. However, existing model-free trajectory-planning policies learn action consequences purely from sampled experience, without exploiting environmental and scenario-specific patterns, leading to slow convergence and short-sighted decision-making. This paper proposes a Multi-horizon Post-Decision State Soft Actor-Critic (MPDS-SAC) algorithm for jointly optimizing multi-UAV trajectories in air-ground collaborative sensing. Within MPDS-SAC, SAC provides the deep reinforcement learning (DRL) backbone for trajectory optimization, while MPDS exploits deterministic transition rules and available observations to analytically predict the states resulting from two successive decisions without additional environment interactions. These additional post-decision states provide the critic with richer training signals, improving sample efficiency and value-estimation accuracy. Experiments on a real-world urban topology show gains over state-of-the-art DRL baselines in reward, convergence speed, and stability, with consistent improvements in information freshness and uplink quality as well as comparable sensor coverage.

引用方法

Q. Zhou, W. Mao, Y. Chen, J. Li, J. Gong, K. Fukuda, and M. Tsukada, “MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing,” in 21st Asian Internet Engineering Conference (AINTEC 2026), Taipei, Taiwan, 2026.

BibTeX

@inproceedings{Zhou2026d,
  title         = {MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing},
  author        = {Quanxi Zhou and Wencan Mao and Yunfei Chen and Junru Li and Jiachen Gong and Kensuke Fukuda and Manabu Tsukada},
  year          = {2026},
  date          = {2026-12-01},
  urldate       = {2026-12-01},
  booktitle     = {21st Asian Internet Engineering Conference (AINTEC 2026)},
  address       = {Taipei, Taiwan},
  abstract      = {Air-ground collaborative sensing, which utilizes unmanned aerial vehicles (UAVs) to collect data from ground wireless sensors, facilitates smart-city monitoring benefiting from mobility, scalability, and fast responses. However, existing model-free trajectory-planning policies learn action consequences purely from sampled experience, without exploiting environmental and scenario-specific patterns, leading to slow convergence and short-sighted decision-making. This paper proposes a Multi-horizon Post-Decision State Soft Actor-Critic (MPDS-SAC) algorithm for jointly optimizing multi-UAV trajectories in air-ground collaborative sensing. Within MPDS-SAC, SAC provides the deep reinforcement learning (DRL) backbone for trajectory optimization, while MPDS exploits deterministic transition rules and available observations to analytically predict the states resulting from two successive decisions without additional environment interactions. These additional post-decision states provide the critic with richer training signals, improving sample efficiency and value-estimation accuracy. Experiments on a real-world urban topology show gains over state-of-the-art DRL baselines in reward, convergence speed, and stability, with consistent improvements in information freshness and uplink quality as well as comparable sensor coverage.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}