LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training
Best Paper Award (Bronze)International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024), Tokyo, Japan, pp. 1–7, 2024年11月28日
arXivDOI: 10.1145/3732437.3732763
3行要約
- LiDARとカメラの特徴を融合し、パノプティックセグメンテーションを行う。
- モデルに2つの変更を加え、動画で学習せずに動画へ適用する。
- 画像・動画のパノプティックセグメンテーション指標が最大5ポイント改善した。
論文の図

アブストラクト(原文)
Panoptic segmentation, which combines instance and semantic segmentation, has gained a lot of attention in autonomous vehicles, due to its comprehensive representation of the scene. This task can be applied to cameras and LiDAR sensors, but there has been a limited focus on combining both sensors to enhance image panoptic segmentation (PS). Although previous research has acknowledged the benefit of 3D data on camera-based scene perception, no specific study has explored the influence of 3D data on image and video panoptic segmentation (VPS). This work seeks to introduce a feature fusion module that enhances PS and VPS by fusing LiDAR and image data for autonomous vehicles. We also illustrate that, in addition to this fusion, our proposed model, which utilizes two simple modifications, can further deliver even more high-quality VPS without being trained on video data. The results demonstrate a substantial improvement in both the image and video panoptic segmentation evaluation metrics by up to 5 points.
関連ニュース
引用方法
F. Ayar, E. Javanmardi, M. Tsukada, M. Javanmardi, and M. Rahmati, “LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training,” in International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024), Tokyo, Japan, 2024, doi: 10.1145/3732437.3732763.
BibTeX
@inproceedings{Ayar2024,
title = {LiDAR-Camera Fusion for Video Panoptic Segmentation without Video Training},
author = {Fardin Ayar and Ehsan Javanmardi and Manabu Tsukada and Mahdi Javanmardi and Mohammad Rahmati},
url = {https://arxiv.org/abs/2412.20881},
doi = {10.1145/3732437.3732763},
year = {2024},
date = {2024-11-28},
urldate = {2024-11-28},
booktitle = {International Conference on Intelligent Computing and its Emerging Applications (ICEA 2024)},
pages = {1--7},
address = {Tokyo, Japan},
abstract = {Panoptic segmentation, which combines instance and semantic segmentation, has gained a lot of attention in autonomous vehicles, due to its comprehensive representation of the scene. This task can be applied to cameras and LiDAR sensors, but there has been a limited focus on combining both sensors to enhance image panoptic segmentation (PS). Although previous research has acknowledged the benefit of 3D data on camera-based scene perception, no specific study has explored the influence of 3D data on image and video panoptic segmentation (VPS). This work seeks to introduce a feature fusion module that enhances PS and VPS by fusing LiDAR and image data for autonomous vehicles. We also illustrate that, in addition to this fusion, our proposed model, which utilizes two simple modifications, can further deliver even more high-quality VPS without being trained on video data. The results demonstrate a substantial improvement in both the image and video panoptic segmentation evaluation metrics by up to 5 points.},
note = {Best Paper Award (Bronze)},
keywords = {Award, ITS},
pubstate = {published},
tppubtype = {inproceedings}
}