Tsukada Lab. Computer Network Group
← Publications Int'l conference

Multi-PrefDrive: Optimizing Large Language Models for Autonomous Driving Through Multi-Preference Tuning

Yun Li, Ehsan Javanmardi, Simon Thompson, Kai Katsumata, Alex Orsholits, Manabu Tsukada

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025), Hangzhou, China, Oct 19, 2025

Summary in three lines

  • Multi-PrefDrive fine-tunes a driving language model using rankings over several candidate actions.
  • A Plackett–Luce preference objective distinguishes rejected actions by the severity of their consequences.
  • CARLA tests report improved driving scores and fewer infrastructure collisions relative to the reported baselines.

Figures from the paper

Figure 1: Multi-PrefDrive architecture
Figure 1. Multi-PrefDrive architectureSource: Figure 1 from Li et al., IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025) (2025) (authors’ figure from the manuscript)The diagram places ranked driving preferences within the model training and decision pipeline.

Abstract

This paper introduces Multi-PrefDrive, a framework that significantly enhances LLM-based autonomous driving through multidimensional preference tuning. Aligning LLMs with human driving preferences is crucial yet challenging, as driving scenarios involve complex decisions where multiple incorrect actions can correspond to a single correct choice. Traditional binary preference tuning fails to capture this complexity. Our approach pairs each chosen action with multiple rejected alternatives, better reflecting real-world driving decisions. By implementing the Plackett-Luce preference model, we enable nuanced ranking of actions across the spectrum of possible errors. Experiments in the CARLA simulator demonstrate that our algorithm achieves an 11.0% improvement in overall score and an 83.6% reduction in infrastructure collisions, while showing perfect compliance with traffic signals in certain environments. Comparative analysis against DPO and its variants reveals that Multi-PrefDrive’s superior discrimination between chosen and rejected actions, which achieving a margin value of 25, and such ability has been directly translates to enhanced driving performance. We implement memory-efficient techniques including LoRA and 4-bit quantization to enable deployment on consumer-grade hardware and will open-source our training code and multi-rejected dataset to advance research in LLM-based autonomous driving systems. Project Page (https://liyun0607.github.io/)

Video

Related video: Multi-PrefDrive: Advancing LLM-Based Autonomous Driving via Multi-Preference Learning

Related research and software

How to cite

Y. Li, E. Javanmardi, S. Thompson, K. Katsumata, A. Orsholits, and M. Tsukada, “Multi-PrefDrive: Optimizing Large Language Models for Autonomous Driving Through Multi-Preference Tuning,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025), Hangzhou, China, 2025, doi: 10.1109/IROS60139.2025.11247608.

BibTeX

@inproceedings{Li2025d,
  title         = {Multi-PrefDrive: Optimizing Large Language Models for Autonomous Driving Through Multi-Preference Tuning},
  author        = {Yun Li and Ehsan Javanmardi and Simon Thompson and Kai Katsumata and Alex Orsholits and Manabu Tsukada},
  url           = {https://liyun0607.github.io/},
  doi           = {10.1109/IROS60139.2025.11247608},
  year          = {2025},
  date          = {2025-10-19},
  urldate       = {2025-10-19},
  booktitle     = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2025)},
  address       = {Hangzhou, China},
  abstract      = {This paper introduces Multi-PrefDrive, a framework that significantly enhances LLM-based autonomous driving through multidimensional preference tuning. Aligning LLMs with human driving preferences is crucial yet challenging, as driving scenarios involve complex decisions where multiple incorrect actions can correspond to a single correct choice. Traditional binary preference tuning fails to capture this complexity. Our approach pairs each chosen action with multiple rejected alternatives, better reflecting real-world driving decisions. By implementing the Plackett-Luce preference model, we enable nuanced ranking of actions across the spectrum of possible errors. Experiments in the CARLA simulator demonstrate that our algorithm achieves an 11.0% improvement in overall score and an 83.6% reduction in infrastructure collisions, while showing perfect compliance with traffic signals in certain environments. Comparative analysis against DPO and its variants reveals that Multi-PrefDrive’s superior discrimination between chosen and rejected actions, which achieving a margin value of 25, and such ability has been directly translates to enhanced driving performance. We implement memory-efficient techniques including LoRA and 4-bit quantization to enable deployment on consumer-grade hardware and will open-source our training code and multi-rejected dataset to advance research in LLM-based autonomous driving systems. Project Page (https://liyun0607.github.io/)},
  keywords      = {CRONOS, ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}