Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality
Honorable Mention AwardIEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025), Saint-Malo, France, Mar 8, 2025
WebsiteDOI: 10.1109/VRW66409.2025.00293
Summary in three lines
- This work adds edge vision processing to a standalone mixed-reality headset.
- Camera frames stream to an AI accelerator for tasks such as object detection, pose estimation and depth estimation.
- Results return to the headset for spatial display, enabling contextual responses within the mixed-reality scene.
Abstract
Spatial computing is evolving towards leveraging data streaming for computationally demanding applications, facilitating a shift to lightweight, untethered, and standalone devices. These devices are ideal candidates for co-processing, where real-time scene context understanding and low-latency data streaming are fundamental for general-purpose Mixed Reality (MR) experiences. This poster demonstrates and evaluates a scalable approach to augmented contextual understanding in MR by implementing edge AI co-processing through a Hailo-8 AI accelerator, a low-power ARM-based single board computer (SBC), and the Magic Leap 2 AR headset. The resulting inferences are streamed back to the headset for spatial reprojection into the user’s vision.
Video
Related research and software
Related news
How to cite
A. Orsholits and M. Tsukada, “Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality,” in IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025), Saint-Malo, France, 2025, doi: 10.1109/VRW66409.2025.00293.
BibTeX
@misc{Orsholits2025b,
title = {Edge Vision AI Co-Processing for Dynamic Context Awareness in Mixed Reality},
author = {Alex Orsholits and Manabu Tsukada},
url = {https://www.youtube.com/watch?v=xxahKZl4K9w https://ieeevr.org/2025/awards/conference-awards/#poster-honorable},
doi = {10.1109/VRW66409.2025.00293},
year = {2025},
date = {2025-03-08},
urldate = {2025-03-08},
booktitle = {IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW 2025)},
address = {Saint-Malo, France},
abstract = {Spatial computing is evolving towards leveraging data streaming for computationally demanding applications, facilitating a shift to lightweight, untethered, and standalone devices. These devices are ideal candidates for co-processing, where real-time scene context understanding and low-latency data streaming are fundamental for general-purpose Mixed Reality (MR) experiences. This poster demonstrates and evaluates a scalable approach to augmented contextual understanding in MR by implementing edge AI co-processing through a Hailo-8 AI accelerator, a low-power ARM-based single board computer (SBC), and the Magic Leap 2 AR headset. The resulting inferences are streamed back to the headset for spatial reprojection into the user’s vision.},
howpublished = {IEEE VR 2025, Poster},
note = {Honorable Mention Award},
keywords = {Award, SDM},
pubstate = {published},
tppubtype = {misc}
}