Tsukada Lab. Computer Network Group

State-Guided Spatial Cross-Attention for Enhanced End-to-End Autonomous Driving

Dongyang Li, Ehsan Javanmardi, Manabu Tsukada

IEEE International Automated Vehicle Validation Conference (IAVVC 2025), Baden-Baden, Germany, 2025年9月30日

3行要約

  • 自車の速度や経路指示に応じて、運転モデルが画像のどこに注目するかを変える。
  • 車両状態から作ったクエリで、現在の操作に役立つ空間特徴を選ぶ。
  • 可視化では高速時に遠方、低速時に近傍への注意が強まった。

論文の図

図 2: 車両状態を使う空間クロスアテンションの構成
図 2. 車両状態を使う空間クロスアテンションの構成出典: Li et al., IEEE International Automated Vehicle Validation Conference (IAVVC 2025) (2025)(著者原稿の図版) の図 2車両状態からクエリを初期化し、前方カメラ画像の情報を選択する過程が分かる。

アブストラクト(原文)

Handling near-accident scenarios is a significant challenge for end-to-end autonomous driving (E2E-AD), as these situations often involve sudden environmental changes, complex interactions with other road users, and high-risk decision-making under uncertainty. Unlike routine driving tasks, near-accident scenarios require rapid and precise responses based on external perception and internal vehicle dynamics. Successfully navigating such situations demands not only a comprehensive understanding of the surrounding environment but also an accurate assessment of the ego vehicle's state, including speed, acceleration, and steering angle, to ensure safe and reliable control. However, conventional E2E-AD models struggle to handle these safety-critical situations effectively. Standard approaches primarily rely on raw sensor inputs to learn driving policies, often overlooking the crucial role of vehicle state information in decision-making. Since many near-accident scenarios involve conditions where the same environmental observation could require vastly different responses depending on the ego vehicle's motion state-such as whether the vehicle is braking, accelerating, or experiencing traction loss-ignoring these internal dynamics can lead to unsafe or suboptimal actions. Furthermore, E2E-AD models typically learn a direct mapping from sensory inputs to control outputs, making it difficult to generalize to highly dynamic and unpredictable interactions, such as emergency evasive maneuvers or sudden braking events. To address these challenges, we propose a state-guided cross-attention mechanism that explicitly models the interaction between the ego vehicle's states and its perception of the environment. By incorporating vehicle state information into the decision-making process, our approach ensures that the model can dynamically adjust its attention to critical sensory inputs based on real-time driving conditions. This allows the autonomous system to make more context-aware decisions, improving its ability to respond effectively to complex and safety-critical scenarios.

引用方法

D. Li, E. Javanmardi, and M. Tsukada, “State-Guided Spatial Cross-Attention for Enhanced End-to-End Autonomous Driving,” in IEEE International Automated Vehicle Validation Conference (IAVVC 2025), Baden-Baden, Germany, 2025, doi: 10.1109/IAVVC61942.2025.11219456.

BibTeX

@inproceedings{Li2025b,
  title         = {State-Guided Spatial Cross-Attention for Enhanced End-to-End Autonomous Driving},
  author        = {Dongyang Li and Ehsan Javanmardi and Manabu Tsukada},
  doi           = {10.1109/IAVVC61942.2025.11219456},
  year          = {2025},
  date          = {2025-09-30},
  urldate       = {2025-09-30},
  booktitle     = {IEEE International Automated Vehicle Validation Conference (IAVVC 2025)},
  address       = {Baden-Baden, Germany},
  abstract      = {Handling near-accident scenarios is a significant challenge for end-to-end autonomous driving (E2E-AD), as these situations often involve sudden environmental changes, complex interactions with other road users, and high-risk decision-making under uncertainty. Unlike routine driving tasks, near-accident scenarios require rapid and precise responses based on external perception and internal vehicle dynamics. Successfully navigating such situations demands not only a comprehensive understanding of the surrounding environment but also an accurate assessment of the ego vehicle's state, including speed, acceleration, and steering angle, to ensure safe and reliable control. However, conventional E2E-AD models struggle to handle these safety-critical situations effectively. Standard approaches primarily rely on raw sensor inputs to learn driving policies, often overlooking the crucial role of vehicle state information in decision-making. Since many near-accident scenarios involve conditions where the same environmental observation could require vastly different responses depending on the ego vehicle's motion state-such as whether the vehicle is braking, accelerating, or experiencing traction loss-ignoring these internal dynamics can lead to unsafe or suboptimal actions. Furthermore, E2E-AD models typically learn a direct mapping from sensory inputs to control outputs, making it difficult to generalize to highly dynamic and unpredictable interactions, such as emergency evasive maneuvers or sudden braking events. To address these challenges, we propose a state-guided cross-attention mechanism that explicitly models the interaction between the ego vehicle's states and its perception of the environment. By incorporating vehicle state information into the decision-making process, our approach ensures that the model can dynamically adjust its attention to critical sensory inputs based on real-time driving conditions. This allows the autonomous system to make more context-aware decisions, improving its ability to respond effectively to complex and safety-critical scenarios.},
  keywords      = {CRONOS, ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}