Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI
Honorable Mention AwardACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, Apr 13, 2026
arXivDOI: 10.1145/3772318.3790537
Summary in three lines
- The study tests vision-language-model personas as a preparation aid for embodied field studies.
- It compares 20 human participants and 20 personas in vehicle–pedestrian interaction, then interviews five researchers.
- The comparison identifies behavior and questionnaire patterns the personas can help explore, as well as limits on reproducing human variation.
Figures from the paper

Abstract
Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.
Video
Related news
How to cite
X. Gui, D. Xia, M. Colley, Y. Li, V. Chauhan, A. Anubhav, Z. Zhou, E. Javanmardi, S. H. Seo, C.-M. Chang, M. Tsukada, and T. Igarashi, “Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI,” in ACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, 2026, doi: 10.1145/3772318.3790537.
BibTeX
@inproceedings{Gui2026,
title = {Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI},
author = {Xinyue Gui and Ding Xia and Mark Colley and Yuan Li and Vishal Chauhan and Anubhav Anubhav and Zhongyi Zhou and Ehsan Javanmardi and Stela Hanbyeol Seo and Chia-Ming Chang and Manabu Tsukada and Takeo Igarashi},
url = {https://arxiv.org/abs/2602.16157 https://youtu.be/PORfzWr2v5c},
doi = {10.1145/3772318.3790537},
year = {2026},
date = {2026-04-13},
urldate = {2026-04-13},
booktitle = {ACM CHI Conference on Human Factors in Computing Systems (CHI 2026)},
address = {Barcelona, Spain},
abstract = {Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.},
note = {Honorable Mention Award},
keywords = {Award, CRONOS, ITS},
pubstate = {published},
tppubtype = {inproceedings}
}