Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Presented at ACM UbiComp / ISWC, New York, NY, USA, vol. 10, no. 2, Jun 15, 2026
Summary in three lines
- The paper explores language models as generators of visible actions for vehicle external interfaces.
- A physical prototype translated model outputs into display and motion cues, and 16 participants compared these with human-designed cues.
- The study found that correctly interpreting a generated action did not always lead to the intended pedestrian response.
Video
How to cite
X. Gui, D. Xia, M. Colley, S. H. Seo, C.-M. Chang, E. Javanmardi, M. Tsukada, and T. Igarashi, “Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Presented at ACM UbiComp / ISWC, vol. 10, no. 2, 2026, doi: 10.1145/3810200.
BibTeX
@article{Gui2026b,
title = {Exploring LLMs for Generating Communicational Actions of External Interfaces on Autonomous Vehicles},
author = {Xinyue Gui and Ding Xia and Mark Colley and Stela Hanbyeol Seo and Chia-Ming Chang and Ehsan Javanmardi and Manabu Tsukada and Takeo Igarashi},
url = {https://youtu.be/S7RZ2ew42ic},
doi = {10.1145/3810200},
year = {2026},
date = {2026-06-15},
urldate = {2026-06-15},
journal = {Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), Presented at ACM UbiComp / ISWC},
volume = {10},
number = {2},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
keywords = {},
pubstate = {published},
tppubtype = {article}
}