Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes
Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab), 2026年3月31日
3行要約
- 粒子ベースのシーンを三角形で直接表現する微分可能レンダラUTriceを開発する。
- 個別の代理形状を構築せず、同じ三角形をレイトレーシングによる最適化とラスタライズに使える。
- 新規視点の画質、描画時間、レイトレーシングで再現する光学効果を評価した。
論文の図


アブストラクト(原文)
Ray tracing of particle-based radiance fields enables physically grounded rendering effects such as depth of field, reflection, refraction, and flexible camera models, which are difficult to achieve with rasterization alone. Recent works, such as ray tracing 3D Gaussian particles, integrate hardware-accelerated ray tracing into novel-view synthesis pipelines. However, these methods rely on proxy geometry to approximate Gaussian primitives, introducing substantial overhead in BVH construction, intersection testing, and memory usage. More fundamentally, Gaussian-based particles are not well suited as unified primitives across rasterization and ray tracing optimization pipelines, leading to approximated representations and redundant computation. In this work, we propose UTrice, a differentiable ray tracing framework that directly uses triangles as rendering primitives for particle-based 3D scene representation. By replacing Gaussian particles and their proxy geometries with differentiable triangles, our method eliminates the need for custom primitive definitions and auxiliary bounding volumes, allowing triangles to be traced natively using modern GPU ray tracing frameworks. This design significantly reduces BVH construction cost and simplifies the rendering pipeline while maintaining compatibility with both ray tracing and rasterization. We introduce a differentiable triangle formulation based on a smooth window function defined in world space, enabling stable gradient propagation from ray-based image formation to triangle geometry, appearance, opacity, and smoothness parameters. This formulation allows triangles to be optimized directly within a ray tracing pipeline using standard backpropagation, without requiring proxy approximations or heuristic gradient re-derivations. Importantly, the proposed window function is invariant under linear transformations, which enables our ray tracer to directly render triangles optimized by rasterization-based methods, such as Triangle Splatting, without any additional preprocessing. This property unifies rasterization-based optimization and ray tracing based rendering within a single primitive representation. We implement our framework using OptiX and custom CUDA kernels, supporting GPU-accelerated BVH construction, k-closest intersection tracing, and front-to-back compositing. The system naturally supports complex ray distributions, making it compatible with non-pinhole camera models and ray tracing based effects. Experimental results on Mip-NeRF 360 and Tanks and Temples datasets demonstrate that our method consistently outperforms Gaussian-based ray tracing approaches in perceptual quality, particularly in preserving high-frequency details and reducing visual artifacts, while maintaining real-time rendering performance. Furthermore, our approach achieves higher generality by unifying particle-based novel-view synthesis, rasterization, and ray tracing under a single triangle-based representation.
引用方法
C. Liu, “Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes,” in Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab), 2026.
BibTeX
@mastersthesis{Liu2026,
title = {Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes},
author = {Changhe Liu},
year = {2026},
date = {2026-03-31},
urldate = {2026-03-31},
school = {Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab)},
abstract = {Ray tracing of particle-based radiance fields enables physically grounded rendering effects such as depth of field, reflection, refraction, and flexible camera models, which are difficult to achieve with rasterization alone. Recent works, such as ray tracing 3D Gaussian particles, integrate hardware-accelerated ray tracing into novel-view synthesis pipelines. However, these methods rely on proxy geometry to approximate Gaussian primitives, introducing substantial overhead in BVH construction, intersection testing, and memory usage. More fundamentally, Gaussian-based particles are not well suited as unified primitives across rasterization and ray tracing optimization pipelines, leading to approximated representations and redundant computation. In this work, we propose UTrice, a differentiable ray tracing framework that directly uses triangles as rendering primitives for particle-based 3D scene representation. By replacing Gaussian particles and their proxy geometries with differentiable triangles, our method eliminates the need for custom primitive definitions and auxiliary bounding volumes, allowing triangles to be traced natively using modern GPU ray tracing frameworks. This design significantly reduces BVH construction cost and simplifies the rendering pipeline while maintaining compatibility with both ray tracing and rasterization. We introduce a differentiable triangle formulation based on a smooth window function defined in world space, enabling stable gradient propagation from ray-based image formation to triangle geometry, appearance, opacity, and smoothness parameters. This formulation allows triangles to be optimized directly within a ray tracing pipeline using standard backpropagation, without requiring proxy approximations or heuristic gradient re-derivations. Importantly, the proposed window function is invariant under linear transformations, which enables our ray tracer to directly render triangles optimized by rasterization-based methods, such as Triangle Splatting, without any additional preprocessing. This property unifies rasterization-based optimization and ray tracing based rendering within a single primitive representation. We implement our framework using OptiX and custom CUDA kernels, supporting GPU-accelerated BVH construction, k-closest intersection tracing, and front-to-back compositing. The system naturally supports complex ray distributions, making it compatible with non-pinhole camera models and ray tracing based effects. Experimental results on Mip-NeRF 360 and Tanks and Temples datasets demonstrate that our method consistently outperforms Gaussian-based ray tracing approaches in perceptual quality, particularly in preserving high-frequency details and reducing visual artifacts, while maintaining real-time rendering performance. Furthermore, our approach achieves higher generality by unifying particle-based novel-view synthesis, rasterization, and ray tracing under a single triangle-based representation.},
keywords = {Thesis},
pubstate = {published},
tppubtype = {mastersthesis}
}