Tsukada Lab. Computer Network Group

co-Sound: An interactive medium with WebAR and spatial synchronization

Kazuma Inokuchi, Manabu Tsukada, Hiroshi Esaki

19th IFIP International Conference on Entertainment Computing (ICEC 2020), Xi'an, China, pp. 255–263, 2020年11月10日

3行要約

  • co-Soundは、音楽イベントのオブジェクトベース収録をブラウザー上のARで再生するアプリケーション。
  • ARオブジェクトを端末間で同期し、複数の視聴者が同じ空間でコンテンツを操作できる。
  • 同期の遅延を測定し、25名の体験者へのアンケートでおおむね好意的な評価を得た。

論文の図

図 1: co-Soundの設計と実装
図 1. co-Soundの設計と実装出典: Kazuma Inokuchi ほか、IFIP ICEC 2020(著者原稿の原図) の図 1複数端末でのAR視聴を支えるメディアと操作情報の流れが分かる。
図 2: co-Soundの再生画面
図 2. co-Soundの再生画面出典: Kazuma Inokuchi ほか、IFIP ICEC 2020(著者原稿の原図) の図 2端末のカメラ映像に音楽のオブジェクトが重なり、操作対象として見える様子が分かる。

アブストラクト(原文)

An Internet-based media service platform can control recording processes and manage video and audio data interconnected by an IP network. Furthermore, the design and implementation of an object-based system for recording enable the flexible playback of the viewing contents. Augmented Reality (AR) is a three-dimensional video projection technology that allows us to interact with both elements in real space and digital space information. However, there are few examples of its use as a method for audio-visual media platforms. In this study, we propose co-Sound, which is an interactive audio-visual playback application for music events, using WebAR. co-Sound was designed as a multimodal interface that dynamically renders object-based AR in response to various actions from viewers on a web browser with low entry costs. Furthermore, by sharing AR objects among multiple devices in real time and bidirectionally, the relationship between users and contents was extended, and interaction among multiple users in the same AR space was possible. We implemented a prototype application, measured the performance of the AR spatial synchronization, and conducted a questionnaire-based evaluation. For subjective evaluation, 25 people experienced co-Sound and completed a questionnaire. We confirmed that the system was developed by object-based method with AR, achieved low-latency synchronization to accept operations from multiple users in real time, and the general acceptance of the system was very high.

動画

引用方法

K. Inokuchi, M. Tsukada, and H. Esaki, “co-Sound: An interactive medium with WebAR and spatial synchronization,” in 19th IFIP International Conference on Entertainment Computing (ICEC 2020), Xi'an, China, 2020, doi: 10.1007/978-3-030-65736-9_22.

BibTeX

@inproceedings{Inokuchi2020b,
  title         = {co-Sound: An interactive medium with WebAR and spatial synchronization},
  author        = {Kazuma Inokuchi and Manabu Tsukada and Hiroshi Esaki},
  url           = {https://hal.archives-ouvertes.fr/hal-02942505/document?.pdf https://youtu.be/Bn1yPlbgqaA},
  doi           = {10.1007/978-3-030-65736-9_22},
  year          = {2020},
  date          = {2020-11-10},
  booktitle     = {19th IFIP International Conference on Entertainment Computing (ICEC 2020)},
  pages         = {255--263},
  publisher     = {Springer, Cham},
  address       = {Xi'an, China},
  abstract      = {An Internet-based media service platform can control recording processes and manage video and audio data interconnected by an IP network. Furthermore, the design and implementation of an object-based system for recording enable the flexible playback of the viewing contents. Augmented Reality (AR) is a three-dimensional video projection technology that allows us to interact with both elements in real space and digital space information. However, there are few examples of its use as a method for audio-visual media platforms. In this study, we propose co-Sound, which is an interactive audio-visual playback application for music events, using WebAR. co-Sound was designed as a multimodal interface that dynamically renders object-based AR in response to various actions from viewers on a web browser with low entry costs. Furthermore, by sharing AR objects among multiple devices in real time and bidirectionally, the relationship between users and contents was extended, and interaction among multiple users in the same AR space was possible. We implemented a prototype application, measured the performance of the AR spatial synchronization, and conducted a questionnaire-based evaluation. For subjective evaluation, 25 people experienced co-Sound and completed a questionnaire. We confirmed that the system was developed by object-based method with AR, achieved low-latency synchronization to accept operations from multiple users in real time, and the general acceptance of the system was very high.},
  keywords      = {SDM},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}