Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes
Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab), Mar 31, 2026
Summary in three lines
- The thesis develops UTrice, a differentiable renderer that represents particle-based scenes directly with triangles.
- The same triangle primitives can be optimized through ray tracing and used in rasterization without constructing separate proxy geometry.
- The evaluation examines novel-view image quality, runtime and ray-traced optical effects.
Figures from the paper


Abstract
Ray tracing of particle-based radiance fields enables physically grounded rendering effects such as depth of field, reflection, refraction, and flexible camera models, which are difficult to achieve with rasterization alone. Recent works, such as ray tracing 3D Gaussian particles, integrate hardware-accelerated ray tracing into novel-view synthesis pipelines. However, these methods rely on proxy geometry to approximate Gaussian primitives, introducing substantial overhead in BVH construction, intersection testing, and memory usage. More fundamentally, Gaussian-based particles are not well suited as unified primitives across rasterization and ray tracing optimization pipelines, leading to approximated representations and redundant computation. In this work, we propose UTrice, a differentiable ray tracing framework that directly uses triangles as rendering primitives for particle-based 3D scene representation. By replacing Gaussian particles and their proxy geometries with differentiable triangles, our method eliminates the need for custom primitive definitions and auxiliary bounding volumes, allowing triangles to be traced natively using modern GPU ray tracing frameworks. This design significantly reduces BVH construction cost and simplifies the rendering pipeline while maintaining compatibility with both ray tracing and rasterization. We introduce a differentiable triangle formulation based on a smooth window function defined in world space, enabling stable gradient propagation from ray-based image formation to triangle geometry, appearance, opacity, and smoothness parameters. This formulation allows triangles to be optimized directly within a ray tracing pipeline using standard backpropagation, without requiring proxy approximations or heuristic gradient re-derivations. Importantly, the proposed window function is invariant under linear transformations, which enables our ray tracer to directly render triangles optimized by rasterization-based methods, such as Triangle Splatting, without any additional preprocessing. This property unifies rasterization-based optimization and ray tracing based rendering within a single primitive representation. We implement our framework using OptiX and custom CUDA kernels, supporting GPU-accelerated BVH construction, k-closest intersection tracing, and front-to-back compositing. The system naturally supports complex ray distributions, making it compatible with non-pinhole camera models and ray tracing based effects. Experimental results on Mip-NeRF 360 and Tanks and Temples datasets demonstrate that our method consistently outperforms Gaussian-based ray tracing approaches in perceptual quality, particularly in preserving high-frequency details and reducing visual artifacts, while maintaining real-time rendering performance. Furthermore, our approach achieves higher generality by unifying particle-based novel-view synthesis, rasterization, and ray tracing under a single triangle-based representation.
How to cite
C. Liu, “Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes,” in Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab), 2026.
BibTeX
@mastersthesis{Liu2026,
title = {Primitive-Unified Differentiable Rendering Method for Particle-Based 3D Scenes},
author = {Changhe Liu},
year = {2026},
date = {2026-03-31},
urldate = {2026-03-31},
school = {Master Thesis, Graduate School of Information Science and Technology, The University of Tokyo (Esaki Lab)},
abstract = {Ray tracing of particle-based radiance fields enables physically grounded rendering effects such as depth of field, reflection, refraction, and flexible camera models, which are difficult to achieve with rasterization alone. Recent works, such as ray tracing 3D Gaussian particles, integrate hardware-accelerated ray tracing into novel-view synthesis pipelines. However, these methods rely on proxy geometry to approximate Gaussian primitives, introducing substantial overhead in BVH construction, intersection testing, and memory usage. More fundamentally, Gaussian-based particles are not well suited as unified primitives across rasterization and ray tracing optimization pipelines, leading to approximated representations and redundant computation. In this work, we propose UTrice, a differentiable ray tracing framework that directly uses triangles as rendering primitives for particle-based 3D scene representation. By replacing Gaussian particles and their proxy geometries with differentiable triangles, our method eliminates the need for custom primitive definitions and auxiliary bounding volumes, allowing triangles to be traced natively using modern GPU ray tracing frameworks. This design significantly reduces BVH construction cost and simplifies the rendering pipeline while maintaining compatibility with both ray tracing and rasterization. We introduce a differentiable triangle formulation based on a smooth window function defined in world space, enabling stable gradient propagation from ray-based image formation to triangle geometry, appearance, opacity, and smoothness parameters. This formulation allows triangles to be optimized directly within a ray tracing pipeline using standard backpropagation, without requiring proxy approximations or heuristic gradient re-derivations. Importantly, the proposed window function is invariant under linear transformations, which enables our ray tracer to directly render triangles optimized by rasterization-based methods, such as Triangle Splatting, without any additional preprocessing. This property unifies rasterization-based optimization and ray tracing based rendering within a single primitive representation. We implement our framework using OptiX and custom CUDA kernels, supporting GPU-accelerated BVH construction, k-closest intersection tracing, and front-to-back compositing. The system naturally supports complex ray distributions, making it compatible with non-pinhole camera models and ray tracing based effects. Experimental results on Mip-NeRF 360 and Tanks and Temples datasets demonstrate that our method consistently outperforms Gaussian-based ray tracing approaches in perceptual quality, particularly in preserving high-frequency details and reducing visual artifacts, while maintaining real-time rendering performance. Furthermore, our approach achieves higher generality by unifying particle-based novel-view synthesis, rasterization, and ray tracing under a single triangle-based representation.},
keywords = {Thesis},
pubstate = {published},
tppubtype = {mastersthesis}
}