Tsukada Lab. Computer Network Group

Large Language Models for Human-Like Autonomous Driving: A Survey

Yun Li, Kai Katsumata, Ehsan Javanmardi, Manabu Tsukada

27th IEEE International Conference on Intelligent Transportation Systems (ITSC 2024), Edmonton, Canada, 2024年9月24日

3行要約

  • 大規模言語モデルを用いる知識ベースの自動運転を、人間らしい運転に向けたアプローチとして整理する。
  • モジュール型とエンドツーエンド型の自動運転での応用を分類した。
  • 実時間推論、安全性の保証、導入費用を課題として挙げる。

論文の図

図 1: 自動運転アプローチの発展
図 1. 自動運転アプローチの発展出典: Li et al., 27th IEEE International Conference on Intelligent Transportation Systems (ITSC 2024), 2024(著者原稿の図版) の図 1ルールベース・学習ベースの自動運転との関係から、大規模言語モデルを使う方式の位置付けが分かる。

アブストラクト(原文)

Large Language Models (LLMs), AI models trained on massive text corpora with remarkable language understanding and generation capabilities, are transforming the field of Autonomous Driving (AD). As AD systems evolve from rule-based and optimization-based methods to learning-based techniques like deep reinforcement learning, they are now poised to embrace a third and more advanced category: knowledge-based AD empowered by LLMs. This shift promises to bring AD closer to human-like AD. However, integrating LLMs into AD systems poses challenges in real-time inference, safety assurance, and deployment costs. This survey provides a comprehensive and critical review of recent progress in leveraging LLMs for AD, focusing on their applications in modular AD pipelines and end- to-end AD systems. We highlight key advancements, identify pressing challenges, and propose promising research directions to bridge the gap between LLMs and AD, thereby facilitating the development of more human-like AD systems. The survey first introduces LLMs’ key features and common training schemes, then delves into their applications in modular AD pipelines and end-to-end AD, respectively, followed by discussions on open challenges and future directions. Through this in-depth analysis, we aim to provide insights and inspiration for researchers and practitioners working at the intersection of AI and autonomous vehicles, ultimately contributing to safer, smarter, and more human-centric AD technologies.

関連ニュース

引用方法

Y. Li, K. Katsumata, E. Javanmardi, and M. Tsukada, “Large Language Models for Human-Like Autonomous Driving: A Survey,” in 27th IEEE International Conference on Intelligent Transportation Systems (ITSC 2024), Edmonton, Canada, 2024, doi: 10.1109/ITSC58415.2024.10919629.

BibTeX

@inproceedings{Yun2024,
  title         = {Large Language Models for Human-Like Autonomous Driving: A Survey},
  author        = {Yun Li and Kai Katsumata and Ehsan Javanmardi and Manabu Tsukada},
  doi           = {10.1109/ITSC58415.2024.10919629},
  url           = {https://arxiv.org/abs/2407.19280},
  eprint        = {2407.19280},
  archiveprefix = {arXiv},
  primaryclass  = {cs.AI},
  year          = {2024},
  date          = {2024-09-24},
  urldate       = {2024-09-24},
  booktitle     = {27th IEEE International Conference on Intelligent Transportation Systems (ITSC 2024)},
  address       = {Edmonton, Canada},
  abstract      = {Large Language Models (LLMs), AI models trained on massive text corpora with remarkable language understanding and generation capabilities, are transforming the field of Autonomous Driving (AD). As AD systems evolve from rule-based and optimization-based methods to learning-based techniques like deep reinforcement learning, they are now poised to embrace a third and more advanced category: knowledge-based AD empowered by LLMs. This shift promises to bring AD closer to human-like AD. However, integrating LLMs into AD systems poses challenges in real-time inference, safety assurance, and deployment costs. This survey provides a comprehensive and critical review of recent progress in leveraging LLMs for AD, focusing on their applications in modular AD pipelines and end- to-end AD systems. We highlight key advancements, identify pressing challenges, and propose promising research directions to bridge the gap between LLMs and AD, thereby facilitating the development of more human-like AD systems. The survey first introduces LLMs’ key features and common training schemes, then delves into their applications in modular AD pipelines and end-to-end AD, respectively, followed by discussions on open challenges and future directions. Through this in-depth analysis, we aim to provide insights and inspiration for researchers and practitioners working at the intersection of AI and autonomous vehicles, ultimately contributing to safer, smarter, and more human-centric AD technologies.},
  keywords      = {ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}