0%
As interest in immersive spatial experiences continues to grow, Tsukada Lab investigates how digital‑twin technology can be leveraged to enable accurate, real‑time 3D audio environments for mixed‑reality applications. This research focuses on cloud-assisted acoustic simulation informed by either large-scale urban geometric models or BIM data, listener positioning, and network-based rendering infrastructure. The goal is to render dynamic spatial audio that behaves realistically with respect to complex geometries and listener movement.
The platform architecture combines large-scale 3D city models or BIM data with RTK-GNSS or SLAM-based tracking systems to determine the listener’s precise location and orientation. This data is fed into spatialization engines that simulate acoustics in real time. Leveraging parallelized server compute for wave-based audio propagation, the system generates soundscapes that reflect environmental features such as occlusion, diffraction, and reverberation. The listener experiences a rich and reactive audio environment that evolves with their movement in physical space.
Several prototypes and field deployments have demonstrated the feasibility of this approach. These include mobile audio visualization systems using smartphones and AR frameworks, remote-rendered immersive urban soundscapes, and real-time synchronization with building-scale acoustic simulations. These implementations validate the technical architecture while providing data to assess simulation accuracy across different environments.
By integrating geospatial infrastructure, embedded sensing, and audio rendering pipelines, the project advances the state of the art in spatial computing and urban informatics. It lays the foundation for a programmable audio infrastructure that can enhance urban navigation, cultural experiences, and situational awareness through immersive, context-aware sound design.
digital twins extended reality
digital twins
autonomous driving machine learning
machine learning v2x
autonomous driving v2x
extended reality
v2x
machine learning v2x
