RealSim-CP: A High-Fidelity Multimodal Cooperative Perception Dataset Bridging the Simulator–Real World Gap
IEEE 103rd Vehicular Technology Conference (VTC2026-Spring), Nice, France, 2026年6月9日
3行要約
- RealSim-CPは日本の道路・交通環境を再現したマルチモーダルの協調認識データセットである。
- DIVPでセンサの物理特性をモデル化し、車両と路側機から同期したカメラ画像とLiDAR点群を生成する。
- 個々の場面を実地収録せず、複数の道路環境で協調認識を検証できる。
論文の図

アブストラクト(原文)
Cooperative perception, which enables vehicles to share sensory information with other vehicles and roadside infrastructure, is essential for advancing autonomous driving beyond single-vehicle limitations. However, existing cooperative perception datasets suffer from two critical limitations: the lack of representation of Japanese traffic environments with their unique characteristics (e.g., left-hand traffic, distinctive vehicle types, and region-specific infrastructure), and the high cost of real-world data collection that constrains dataset scale and diversity. To address these challenges, we present RealSim-CP, a novel cooperative perception dataset generated using the Driving Intelligence Validation Platform (DIVP), a physics-based simulation system that employs ray tracing and electromagnetic-wave modeling to produce highly realistic sensor data comparable to real-world quality. Leveraging DIVP’s high-fidelity simulation capabilities, we efficiently generate a large-scale dataset comprising synchronized multimodal data—camera images and LiDAR point clouds—from multiple cooperative agents, including vehicles and roadside units. The dataset covers three urban maps representing Tokyo regions (Aomi, Odaiba, and the Shutoko Expressway) under diverse environmental conditions, including clear daytime, rainy daytime, and clear nighttime scenarios. All data are provided in the standardized OpenLABEL format with annotations for 12 object classes, totaling 140k images and 30k point clouds. We further evaluate RealSim-CP using CoopDet3D, a state-of-the-art multimodal cooperative 3D object detection framework, demonstrating the effectiveness of the dataset for advanced cooperative perception research. These results indicate that high-fidelity simulation can effectively bridge the gap between simulation and real-world deployment while significantly reducing data collection costs. RealSim-CP provides the first region-specific cooperative perception dataset tailored to Japanese traffic environments.
コード · データ
関連する研究・ソフトウェア
引用方法
Y. Aizono, E. Javanmardi, F. Ayar, M. Javanmardi, M. Tsukada, and H. Esaki, “RealSim-CP: A High-Fidelity Multimodal Cooperative Perception Dataset Bridging the Simulator–Real World Gap,” in IEEE 103rd Vehicular Technology Conference (VTC2026-Spring), Nice, France, 2026.
BibTeX
@inproceedings{Aizono2026,
title = {RealSim-CP: A High-Fidelity Multimodal Cooperative Perception Dataset Bridging the Simulator–Real World Gap},
author = {Yuji Aizono and Ehsan Javanmardi and Fardin Ayar and Mahdi Javanmardi and Manabu Tsukada and Hiroshi Esaki},
url = {https://github.com/tlab-wide/realsim-cp https://tlab-wide.github.io/realsim-cp/},
year = {2026},
date = {2026-06-09},
urldate = {2026-06-09},
booktitle = {IEEE 103rd Vehicular Technology Conference (VTC2026-Spring)},
address = {Nice, France},
abstract = {
Cooperative perception, which enables vehicles to share sensory information with other vehicles and roadside infrastructure, is essential for advancing autonomous driving beyond single-vehicle limitations. However, existing cooperative perception datasets suffer from two critical limitations: the lack of representation of Japanese traffic environments with their unique characteristics (e.g., left-hand traffic, distinctive vehicle types, and region-specific infrastructure), and the high cost of real-world data collection that constrains dataset scale and diversity.
To address these challenges, we present RealSim-CP, a novel cooperative perception dataset generated using the Driving Intelligence Validation Platform (DIVP), a physics-based simulation system that employs ray tracing and electromagnetic-wave modeling to produce highly realistic sensor data comparable to real-world quality. Leveraging DIVP’s high-fidelity simulation capabilities, we efficiently generate a large-scale dataset comprising synchronized multimodal data—camera images and LiDAR point clouds—from multiple cooperative agents, including vehicles and roadside units. The dataset covers three urban maps representing Tokyo regions (Aomi, Odaiba, and the Shutoko Expressway) under diverse environmental conditions, including clear daytime, rainy daytime, and clear nighttime scenarios.
All data are provided in the standardized OpenLABEL format with annotations for 12 object classes, totaling 140k images and 30k point clouds. We further evaluate RealSim-CP using CoopDet3D, a state-of-the-art multimodal cooperative 3D object detection framework, demonstrating the effectiveness of the dataset for advanced cooperative perception research. These results indicate that high-fidelity simulation can effectively bridge the gap between simulation and real-world deployment while significantly reducing data collection costs. RealSim-CP provides the first region-specific cooperative perception dataset tailored to Japanese traffic environments.
},
keywords = {CRONOS, ITS},
pubstate = {published},
tppubtype = {inproceedings}
}