Tsukada Lab. Computer Network Group
← Publications Int'l conference

Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI

Honorable Mention Award

Xinyue Gui, Ding Xia, Mark Colley, Yuan Li, Vishal Chauhan, Anubhav Anubhav, Zhongyi Zhou, Ehsan Javanmardi, Stela Hanbyeol Seo, Chia-Ming Chang, Manabu Tsukada, Takeo Igarashi

ACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, Apr 13, 2026

Summary in three lines

  • The study tests vision-language-model personas as a preparation aid for embodied field studies.
  • It compares 20 human participants and 20 personas in vehicle–pedestrian interaction, then interviews five researchers.
  • The comparison identifies behavior and questionnaire patterns the personas can help explore, as well as limits on reproducing human variation.

Figures from the paper

Figure 1: Overview of the human and VLM persona studies
Figure 1. Overview of the human and VLM persona studiesSource: Figure 1 from Gui et al., ACM CHI Conference on Human Factors in Computing Systems (CHI 2026) (2026) (authors’ arXiv version 2602.16157, CC BY-SA 4.0)The study design separates data collection, comparison, and interviews about using personas to prepare field studies.

Abstract

Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.

Video

Related news

How to cite

X. Gui, D. Xia, M. Colley, Y. Li, V. Chauhan, A. Anubhav, Z. Zhou, E. Javanmardi, S. H. Seo, C.-M. Chang, M. Tsukada, and T. Igarashi, “Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI,” in ACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, 2026, doi: 10.1145/3772318.3790537.

BibTeX

@inproceedings{Gui2026,
  title         = {Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI},
  author        = {Xinyue Gui and Ding Xia and Mark Colley and Yuan Li and Vishal Chauhan and Anubhav Anubhav and Zhongyi Zhou and Ehsan Javanmardi and Stela Hanbyeol Seo and Chia-Ming Chang and Manabu Tsukada and Takeo Igarashi},
  url           = {https://arxiv.org/abs/2602.16157 https://youtu.be/PORfzWr2v5c},
  doi           = {10.1145/3772318.3790537},
  year          = {2026},
  date          = {2026-04-13},
  urldate       = {2026-04-13},
  booktitle     = {ACM CHI Conference on Human Factors in Computing Systems (CHI 2026)},
  address       = {Barcelona, Spain},
  abstract      = {Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.},
  note          = {Honorable Mention Award},
  keywords      = {Award, CRONOS, ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}