Tsukada Lab. Computer Network Group

Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality

Honorable Mention Award

Alex Orsholits, Manabu Tsukada

IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025), Saint-Malo, France, Mar 8, 2025

Summary in three lines

  • This work adds edge vision processing to a standalone mixed-reality headset.
  • Camera frames stream to an AI accelerator for tasks such as object detection, pose estimation and depth estimation.
  • Results return to the headset for spatial display, enabling contextual responses within the mixed-reality scene.

Abstract

Spatial computing is evolving towards leveraging data streaming for computationally demanding applications, facilitating a shift to lightweight, untethered, and standalone devices. These devices are ideal candidates for co-processing, where real-time scene context understanding and low-latency data streaming are fundamental for general-purpose Mixed Reality (MR) experiences. This poster demonstrates and evaluates a scalable approach to augmented contextual understanding in MR by implementing edge AI co-processing through a Hailo-8 AI accelerator, a low-power ARM-based single board computer (SBC), and the Magic Leap 2 AR headset. The resulting inferences are streamed back to the headset for spatial reprojection into the user’s vision.

Video

Related research and software

Related news

How to cite

A. Orsholits and M. Tsukada, “Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality,” in IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025), Saint-Malo, France, 2025, doi: 10.1109/VRW66409.2025.00293.

BibTeX

@misc{Orsholits2025b,
  title         = {Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality},
  author        = {Alex Orsholits and Manabu Tsukada},
  url           = {https://www.youtube.com/watch?v=xxahKZl4K9w https://ieeevr.org/2025/awards/conference-awards/#poster-honorable},
  doi           = {10.1109/VRW66409.2025.00293},
  year          = {2025},
  date          = {2025-03-08},
  urldate       = {2025-03-08},
  booktitle     = {IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025)},
  address       = {Saint-Malo, France},
  abstract      = {Spatial computing is evolving towards leveraging data streaming for computationally demanding applications, facilitating a shift to lightweight, untethered, and standalone devices. These devices are ideal candidates for co-processing, where real-time scene context understanding and low-latency data streaming are fundamental for general-purpose Mixed Reality (MR) experiences. This poster demonstrates and evaluates a scalable approach to augmented contextual understanding in MR by implementing edge AI co-processing through a Hailo-8 AI accelerator, a low-power ARM-based single board computer (SBC), and the Magic Leap 2 AR headset. The resulting inferences are streamed back to the headset for spatial reprojection into the user’s vision.},
  howpublished  = {IEEE VR 2025, Poster},
  note          = {Honorable Mention Award},
  keywords      = {Award, SDM},
  pubstate      = {published},
  tppubtype     = {misc}
}