AWSIM-VR: A Tightly-Coupled Virtual Reality Extension for Human-in-the-Loop Pedestrian-Autonomous Vehicle Interaction
3rd Annual IEEE International Conference on Metaverse Computing, Networking, and Applications (MetaCom 2025), Seoul, Korea, Aug 27, 2025
GitHub (code)DOI: 10.1109/MetaCom65502.2025.00063
Summary in three lines
- AWSIM-VR brings a real pedestrian into a closed-loop autonomous-vehicle simulation through virtual reality.
- Pedestrian motion feeds the driving stack, and changes in simulated vehicle behavior are visible to the participant.
- A user study demonstrates evaluation of a vehicle display and a light-strip interface.
Abstract
Effective communication between autonomous vehicles (AVs) and pedestrians is crucial for ensuring future urban traffic safety. While external Human-Machine Interfaces (eHMIs) have emerged as promising solutions, current evaluation methodologies — particularly Virtual Reality (VR)-based studies — typically rely on scripted or pre-defined autonomous vehicle behaviors, limiting realism and neglecting pedestrians' active role in interactions. To address this, we introduce AWSIM-VR, a tightly-coupled VR extension of the AWSIM autonomous driving simulator, enabling real-time, human-in-the-loop pedestrian-AV interactions by directly integrating unmodified, real autonomous driving software (Autoware) into the simulation loop. Unlike previous systems, AWSIM-VR provides authentic, bidirectional interaction: pedestrians' actions dynamically influence vehicle decision-making and eHMI responses in real-time, closely emulating real-world AV scenarios. In user studies directly comparing AWSIM-VR to existing methodologies, participants reported significantly higher perceived realism and immersion, underscoring the importance of authentic autonomous behaviors in VR-based pedestrian interaction research. By directly utilizing production-level autonomous driving stacks, AWSIM-VR represents a significant methodological advancement, enabling more realistic, effective, and safer development and evaluation of eHMIs and AV technologies.
Code
Related news
How to cite
S. Zhang, A. Orsholits, E. Javanmardi, and M. Tsukada, “AWSIM-VR: A Tightly-Coupled Virtual Reality Extension for Human-in-the-Loop Pedestrian-Autonomous Vehicle Interaction,” in 3rd Annual IEEE International Conference on Metaverse Computing, Networking, and Applications (MetaCom 2025), Seoul, Korea, 2025, doi: 10.1109/MetaCom65502.2025.00063.
BibTeX
@inproceedings{Zhang2025,
title = {AWSIM-VR: A Tightly-Coupled Virtual Reality Extension for Human-in-the-Loop Pedestrian-Autonomous Vehicle Interaction},
author = {Shangkai Zhang and Alex Orsholits and Ehsan Javanmardi and Manabu Tsukada},
url = {https://github.com/zhangshangkai/AWSIM-VR},
doi = {10.1109/MetaCom65502.2025.00063},
year = {2025},
date = {2025-08-27},
urldate = {2025-08-27},
booktitle = {3rd Annual IEEE International Conference on Metaverse Computing, Networking, and Applications (MetaCom 2025)},
address = {Seoul, Korea},
abstract = {Effective communication between autonomous vehicles (AVs) and pedestrians is crucial for ensuring future urban traffic safety. While external Human-Machine Interfaces (eHMIs) have emerged as promising solutions, current evaluation methodologies — particularly Virtual Reality (VR)-based studies — typically rely on scripted or pre-defined autonomous vehicle behaviors, limiting realism and neglecting pedestrians' active role in interactions. To address this, we introduce AWSIM-VR, a tightly-coupled VR extension of the AWSIM autonomous driving simulator, enabling real-time, human-in-the-loop pedestrian-AV interactions by directly integrating unmodified, real autonomous driving software (Autoware) into the simulation loop. Unlike previous systems, AWSIM-VR provides authentic, bidirectional interaction: pedestrians' actions dynamically influence vehicle decision-making and eHMI responses in real-time, closely emulating real-world AV scenarios. In user studies directly comparing AWSIM-VR to existing methodologies, participants reported significantly higher perceived realism and immersion, underscoring the importance of authentic autonomous behaviors in VR-based pedestrian interaction research. By directly utilizing production-level autonomous driving stacks, AWSIM-VR represents a significant methodological advancement, enabling more realistic, effective, and safer development and evaluation of eHMIs and AV technologies.},
keywords = {CRONOS, ITS},
pubstate = {published},
tppubtype = {inproceedings}
}