Tsukada Lab. Computer Network Group
← Publications Int'l conference

Cross-Attention Enhanced Imitation Learning for End-to-end Autonomous Driving in Unprotected Turns

Best Paper Award (Silver)

Dongyang Li, Ehsan Javanmardi, Naren Bao, Manabu Tsukada

International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024), Tokyo, Japan, pp. 81–86, Nov 28, 2024

Summary in three lines

  • An unprotected turn depends on both the surrounding traffic and the ego vehicle’s state.
  • A cross-attention layer combines image and vehicle-state features for imitation learning.
  • In AWSIM, the approach outperformed conventional imitation learning with limited demonstrations.

Figures from the paper

Figure 3: Cross-attention model for unprotected turns
Figure 3. Cross-attention model for unprotected turnsSource: Figure 3 from Li et al., International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024), 2024 (original artwork from the authors’ manuscript)The model encodes images and ego state separately, joins them through cross-attention, and outputs driving commands.

Abstract

Performing an unprotected turn in the intersection is a complex scenario for autonomous vehicles. It not only requires a comprehensive understanding of the surrounding environment but also highly relies on the ego vehicle’s current state to make safe decisions. A conventional way to learn end-to-end autonomous driving is imitation learning, which is learning from expert demonstrations. While most imitation learning methods focus on imitating the expert action, they often fail to imitate a complex policy efficiently when the ego vehicle’s states are crucial to the scenario because there might be arbitrary optimal actions under different states. To address this issue and investigate how vehicle states affect autonomous driving, we present a novel cross-attention enhanced imitation learning approach for end-to-end autonomous driving in unprotected turns, focusing on capturing the relationships between the ego vehicle’s states and its perception of the environment. We evaluate our model in AWSIM, an open-source autonomous driving simulator, and the results demonstrate that our model outperformed conventional imitation learning-based baselines in performing unprotected turn scenarios, showcasing its ability to imitate a complex policy efficiently.

Related news

How to cite

D. Li, E. Javanmardi, N. Bao, and M. Tsukada, “Cross-Attention Enhanced Imitation Learning for End-to-end Autonomous Driving in Unprotected Turns,” in International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024), Tokyo, Japan, 2024, doi: 10.1145/3732437.3732764.

BibTeX

@inproceedings{Li2024,
  title         = {Cross-Attention Enhanced Imitation Learning for End-to-end Autonomous Driving in Unprotected Turns},
  author        = {Dongyang Li and Ehsan Javanmardi and Naren Bao and Manabu Tsukada},
  doi           = {10.1145/3732437.3732764},
  year          = {2024},
  date          = {2024-11-28},
  urldate       = {2024-11-28},
  booktitle     = {International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024)},
  pages         = {81--86},
  address       = {Tokyo, Japan},
  abstract      = {Performing an unprotected turn in the intersection is a complex scenario for autonomous vehicles. It not only requires a comprehensive understanding of the surrounding environment but also highly relies on the ego vehicle’s current state to make safe decisions. A conventional way to learn end-to-end autonomous driving is imitation learning, which is learning from expert demonstrations. While most imitation learning methods focus on imitating the expert action, they often fail to imitate a complex policy efficiently when the ego vehicle’s states are crucial to the scenario because there might be arbitrary optimal actions under different states. To address this issue and investigate how vehicle states affect autonomous driving, we present a novel cross-attention enhanced imitation learning approach for end-to-end autonomous driving in unprotected turns, focusing on capturing the relationships between the ego vehicle’s states and its perception of the environment. We evaluate our model in AWSIM, an open-source autonomous driving simulator, and the results demonstrate that our model outperformed conventional imitation learning-based baselines in performing unprotected turn scenarios, showcasing its ability to imitate a complex policy efficiently.},
  note          = {Best Paper Award (Silver)},
  keywords      = {Award, ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}