Tsukada Lab. Computer Network Group
← Publications Int'l conference

MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing

Quanxi Zhou, Wencan Mao, Yunfei Chen, Junru Li, Jiachen Gong, Kensuke Fukuda, Manabu Tsukada

21st Asian Internet Engineering Conference (AINTEC 2026), Taipei, Taiwan, Dec 1, 2026

Summary in three lines

  • MPDS-SAC plans UAV trajectories to collect measurements from urban ground sensors.
  • It trains the critic using predictable consequences of actions at two horizons without requiring additional interactions.
  • In a city simulation with eight UAVs, it improved information freshness and converged faster than SAC and TD3.

Figures from the paper

Figure 1: Urban air–ground sensing system
Figure 1. Urban air–ground sensing systemSource: Figure 1 from Zhou et al., 21st Asian Internet Engineering Conference (AINTEC 2026) (2026) (authors’ figure from the manuscript)The arrangement shows UAVs collecting data from ground sensors and communicating with base stations.
Figure 3: Training rewards across policies
Figure 3. Training rewards across policiesSource: Figure 3 from Zhou et al., 21st Asian Internet Engineering Conference (AINTEC 2026) (2026) (authors’ figure from the manuscript)The learning curves show how MPDS-SAC reaches high reward earlier than the reference policies in the simulation.

Abstract

Air-ground collaborative sensing, which utilizes unmanned aerial vehicles (UAVs) to collect data from ground wireless sensors, facilitates smart-city monitoring benefiting from mobility, scalability, and fast responses. However, existing model-free trajectory-planning policies learn action consequences purely from sampled experience, without exploiting environmental and scenario-specific patterns, leading to slow convergence and short-sighted decision-making. This paper proposes a Multi-horizon Post-Decision State Soft Actor-Critic (MPDS-SAC) algorithm for jointly optimizing multi-UAV trajectories in air-ground collaborative sensing. Within MPDS-SAC, SAC provides the deep reinforcement learning (DRL) backbone for trajectory optimization, while MPDS exploits deterministic transition rules and available observations to analytically predict the states resulting from two successive decisions without additional environment interactions. These additional post-decision states provide the critic with richer training signals, improving sample efficiency and value-estimation accuracy. Experiments on a real-world urban topology show gains over state-of-the-art DRL baselines in reward, convergence speed, and stability, with consistent improvements in information freshness and uplink quality as well as comparable sensor coverage.

How to cite

Q. Zhou, W. Mao, Y. Chen, J. Li, J. Gong, K. Fukuda, and M. Tsukada, “MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing,” in 21st Asian Internet Engineering Conference (AINTEC 2026), Taipei, Taiwan, 2026.

BibTeX

@inproceedings{Zhou2026d,
  title         = {MPDS-SAC: Multi-horizon Post-Decision State Soft Actor-Critic for Air-Ground Collaborative Sensing},
  author        = {Quanxi Zhou and Wencan Mao and Yunfei Chen and Junru Li and Jiachen Gong and Kensuke Fukuda and Manabu Tsukada},
  year          = {2026},
  date          = {2026-12-01},
  urldate       = {2026-12-01},
  booktitle     = {21st Asian Internet Engineering Conference (AINTEC 2026)},
  address       = {Taipei, Taiwan},
  abstract      = {Air-ground collaborative sensing, which utilizes unmanned aerial vehicles (UAVs) to collect data from ground wireless sensors, facilitates smart-city monitoring benefiting from mobility, scalability, and fast responses. However, existing model-free trajectory-planning policies learn action consequences purely from sampled experience, without exploiting environmental and scenario-specific patterns, leading to slow convergence and short-sighted decision-making. This paper proposes a Multi-horizon Post-Decision State Soft Actor-Critic (MPDS-SAC) algorithm for jointly optimizing multi-UAV trajectories in air-ground collaborative sensing. Within MPDS-SAC, SAC provides the deep reinforcement learning (DRL) backbone for trajectory optimization, while MPDS exploits deterministic transition rules and available observations to analytically predict the states resulting from two successive decisions without additional environment interactions. These additional post-decision states provide the critic with richer training signals, improving sample efficiency and value-estimation accuracy. Experiments on a real-world urban topology show gains over state-of-the-art DRL baselines in reward, convergence speed, and stability, with consistent improvements in information freshness and uplink quality as well as comparable sensor coverage.},
  keywords      = {Other},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}