Tsukada Lab. Computer Network Group
← Publications Int'l conference

LiVRation: Remote VR live platform with interactive 3D audio-visual service

Takashi Kasuya, Manabu Tsukada, Yu Komohara, Shigeki Takasaka, Takuhiro Mizuno, Yoshitaka Nomura, Yuta Ueda, Hiroshi Esaki

IEEE Games Entertainment & Media Conference (GEM 2019), Yale University, New Haven, CT, USA, pp. 1–7, Jun 19, 2019

Summary in three lines

  • LiVRation lets viewers choose where to watch and listen within a recorded concert environment.
  • The system represents the music and visuals as objects that can be rendered interactively in three dimensions.
  • The authors surveyed 211 participants, and a demonstration won the first prize at Billboard Live Hackasong 2017.

Figures from the paper

Figure 1: Microphone and camera arrangement for the live recording
Figure 1. Microphone and camera arrangement for the live recordingSource: Figure 1 from Kasuya et al., IEEE GEM 2019 (original figure from the authors’ manuscript)The diagram shows how the team captured sound sources and 360° video for later reconstruction of the performance.
Figure 6: Subjective evaluation of LiVRation
Figure 6. Subjective evaluation of LiVRationSource: Figure 6 from Kasuya et al., IEEE GEM 2019 (original figure from the authors’ manuscript)The chart summarizes viewer responses about spatial audio, video and the interactive experience.

Abstract

Of late, various audio-visual services based on the internet are being deployed extensively. Among these, object- based audio-visual services are attracting more attention. In 2014, we had established the software defined media (SDM) consortium to investigate object-based and internet-based audio- visual services. Despite the increasing demand and popularity of live concert events, the placement of the microphone and camera limit the free-viewpoint watching of the contents of package media, such as DVDs. In this study, we design and implement an interactive 3D audio-visual service system called LiVRation, with a free-view-listen point. For subjective evaluation, 211 people were made to experience LiVRation and answer a questionnaire, subsequently. In addition, we demonstrated the system in the ”Billboard Live Hackasong 2017” hosted by Billboard Japan and received the first prize, based on the votes of the judges as well as the audience.

Video

Related research and software

How to cite

T. Kasuya, M. Tsukada, Y. Komohara, S. Takasaka, T. Mizuno, Y. Nomura, Y. Ueda, and H. Esaki, “LiVRation: Remote VR live platform with interactive 3D audio-visual service,” in IEEE Games Entertainment & Media Conference (GEM 2019), Yale University, New Haven, CT, USA, 2019, doi: 10.1109/GEM.2019.8811549.

BibTeX

@inproceedings{Kasuya2019,
  title         = {LiVRation: Remote VR live platform with interactive 3D audio-visual service},
  author        = {Takashi Kasuya and Manabu Tsukada and Yu Komohara and Shigeki Takasaka and Takuhiro Mizuno and Yoshitaka Nomura and Yuta Ueda and Hiroshi Esaki},
  url           = {https://hal.archives-ouvertes.fr/hal-02136247/document?.pdf https://youtu.be/4MElM4_t2Io},
  doi           = {10.1109/GEM.2019.8811549},
  year          = {2019},
  date          = {2019-06-19},
  booktitle     = {IEEE Games Entertainment & Media Conference (GEM 2019)},
  pages         = {1--7},
  address       = {Yale University, New Haven, CT, USA},
  abstract      = {Of late, various audio-visual services based on the internet are being deployed extensively. Among these, object- based audio-visual services are attracting more attention. In 2014, we had established the software defined media (SDM) consortium to investigate object-based and internet-based audio- visual services. Despite the increasing demand and popularity of live concert events, the placement of the microphone and camera limit the free-viewpoint watching of the contents of package media, such as DVDs. In this study, we design and implement an interactive 3D audio-visual service system called LiVRation, with a free-view-listen point. For subjective evaluation, 211 people were made to experience LiVRation and answer a questionnaire, subsequently. In addition, we demonstrated the system in the ”Billboard Live Hackasong 2017” hosted by Billboard Japan and received the first prize, based on the votes of the judges as well as the audience.},
  keywords      = {SDM},
  pubstate      = {published},
  tppubtype     = {inproceedings}
}