Tsukada Lab. Computer Network Group

An Ontology for Spatio-Temporal Media Management and an Interactive Application

Takuro Sone, Shin Kato, Ray Atarashi, Jin Nakazato, Manabu Tsukada, Hiroshi Esaki

Future Internet, vol. 15, no. 225, Jun 23, 2023

Summary in three lines

  • The SDM Ontology describes media, recording and playback equipment, and related time and location information.
  • The authors modeled recordings of classical and jazz concerts as linked open data.
  • The Web360² application uses that data to let viewers interact with the recordings.

Figures from the paper

Figure 10: Web360² system design
Figure 10. Web360² system designSource: Figure 10 from Sone et al., Future Internet, 2023 (original artwork from the authors’ manuscript)The application queries the linked dataset to determine which media and associated metadata to load for playback.

Abstract

In addition to traditional viewing media, metadata that record the physical space from multiple perspectives will become extremely important in realizing interactive applications such as Virtual Reality(VR), Augmented Reality(AR). This paper proposes the Software Defined Media (SDM) Ontology designed to describe spatio-temporal media and the systems that handle them comprehensively. Spatio-temporal media refers to video, audio, and various sensor values recorded together with time and location information. The SDM Ontology can flexibly and precisely represent spatio-temporal media, equipment, and functions that record, process, edit, and play them and related semantic information. In addition, we recorded classical and jazz concerts using many video cameras and audio microphones, and then processed and edited the video and audio data with related metadata. Then, we created a dataset using the SDM Ontology and published it as linked open data(LOD). Furthermore, we developed "Web360^2" an application that enables users to interactively view and experience 360-degree video and spatial acoustic sounds by referring to this dataset. We conducted a subjective evaluation by using a user questionnaire. Web360^2 is a data-driven web application that obtains video and audio data and related metadata by querying the Dataset.

Code

Related research and software

How to cite

T. Sone, S. Kato, R. Atarashi, J. Nakazato, M. Tsukada, and H. Esaki, “An Ontology for Spatio-Temporal Media Management and an Interactive Application,” Future Internet, vol. 15, no. 225, 2023, doi: 10.3390/fi15070225.

BibTeX

@article{Sone2023,
  title         = {An Ontology for Spatio-Temporal Media Management and an Interactive Application},
  author        = {Takuro Sone and Shin Kato and Ray Atarashi and Jin Nakazato and Manabu Tsukada and Hiroshi Esaki},
  url           = {https://github.com/sdm-wg/web360square-vue https://tlab.hongo.wide.ad.jp/sdmo/},
  doi           = {10.3390/fi15070225},
  issn          = {1999-5903},
  year          = {2023},
  date          = {2023-06-23},
  urldate       = {2023-06-23},
  journal       = {Future Internet},
  volume        = {15},
  number        = {225},
  issue         = {7},
  abstract      = {In addition to traditional viewing media, metadata that record the physical space from multiple perspectives will become extremely important in realizing interactive applications such as Virtual Reality(VR), Augmented Reality(AR). This paper proposes the Software Defined Media (SDM) Ontology designed to describe spatio-temporal media and the systems that handle them comprehensively. Spatio-temporal media refers to video, audio, and various sensor values recorded together with time and location information. The SDM Ontology can flexibly and precisely represent spatio-temporal media, equipment, and functions that record, process, edit, and play them and related semantic information. In addition, we recorded classical and jazz concerts using many video cameras and audio microphones, and then processed and edited the video and audio data with related metadata. Then, we created a dataset using the SDM Ontology and published it as linked open data(LOD). Furthermore, we developed "Web360^2" an application that enables users to interactively view and experience 360-degree video and spatial acoustic sounds by referring to this dataset. We conducted a subjective evaluation by using a user questionnaire. Web360^2 is a data-driven web application that obtains video and audio data and related metadata by querying the Dataset.},
  keywords      = {SDM, Software-Data},
  pubstate      = {published},
  tppubtype     = {article}
}