Tsukada Lab. Computer Network Group

Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI

Honorable Mention Award

Xinyue Gui, Ding Xia, Mark Colley, Yuan Li, Vishal Chauhan, Anubhav Anubhav, Zhongyi Zhou, Ehsan Javanmardi, Stela Hanbyeol Seo, Chia-Ming Chang, Manabu Tsukada, Takeo Igarashi

ACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, 2026年4月13日

3行要約

  • 視覚言語モデルによるペルソナが、身体を伴う現地調査の準備に役立つかを調べる。
  • 車両と歩行者の場面で、人間20人とペルソナ20体を比較し、研究者5人にもインタビューした。
  • 行動・回答パターンを探索する用途と、人間のばらつきを再現する際の限界を整理した。

論文の図

図 1: 人とVLMペルソナの比較実験
図 1. 人とVLMペルソナの比較実験出典: Gui et al., ACM CHI Conference on Human Factors in Computing Systems (CHI 2026) (2026)(arXiv著者版 2602.16157、CC BY-SA 4.0) の図 1人とペルソナのデータ収集、比較、現地調査への利用を尋ねるインタビューの関係を示す。

アブストラクト(原文)

Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.

動画

関連ニュース

引用方法

X. Gui, D. Xia, M. Colley, Y. Li, V. Chauhan, A. Anubhav, Z. Zhou, E. Javanmardi, S. H. Seo, C.-M. Chang, M. Tsukada, and T. Igarashi, “Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI,” in ACM CHI Conference on Human Factors in Computing Systems (CHI 2026), Barcelona, Spain, 2026, doi: 10.1145/3772318.3790537.

BibTeX

@inproceedings{Gui2026,
  title         = {Peeking Ahead of the Field Study: Exploring VLM Personas as Support Tools for Embodied Studies in HCI},
  author        = {Xinyue Gui and Ding Xia and Mark Colley and Yuan Li and Vishal Chauhan and Anubhav Anubhav and Zhongyi Zhou and Ehsan Javanmardi and Stela Hanbyeol Seo and Chia-Ming Chang and Manabu Tsukada and Takeo Igarashi},
  url           = {https://arxiv.org/abs/2602.16157 https://youtu.be/PORfzWr2v5c},
  doi           = {10.1145/3772318.3790537},
  year          = {2026},
  date          = {2026-04-13},
  urldate       = {2026-04-13},
  booktitle     = {ACM CHI Conference on Human Factors in Computing Systems (CHI 2026)},
  address       = {Barcelona, Spain},
  abstract      = {Field studies are irreplaceable but costly, time-consuming, and error-prone, which need careful preparation. Inspired by rapid-prototyping in manufacturing, we propose a fast, low-cost evaluation method using Vision-Language Model (VLM) personas to simulate outcomes comparable to field results. While LLMs show human-like reasoning and language capabilities, autonomous vehicle (AV)-pedestrian interaction requires spatial awareness, emotional empathy, and behavioral generation. This raises our research question: To what extent can VLM personas mimic human responses in field studies? We conducted parallel studies: 1) one real-world study with 20 participants, and 2) one video-study using 20 VLM personas, both on a street-crossing task. We compared their responses and interviewed five HCI researchers on potential applications. Results show that VLM personas mimic human response patterns (e.g., average crossing times of 5.25 s vs. 5.07 s) lack the behavioral variability and depth. They show promise for formative studies, field study preparation, and human data augmentation.},
  note          = {Honorable Mention Award},
  keywords      = {Award, CRONOS, ITS},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}