Image-Based 3D Representations for Real-World Novel View Synthesis

Abstract

This dissertation studies image-based three-dimensional (3D) representation meth- ods for real-world novel view synthesis. Recent advances such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have enabled high-quality reconstruction and rendering from posed images and videos. However, real-world scenes still present several challenges that are not fully addressed by standard formulations. In particular, practical 3D representations should handle spatially unbounded scenes, dynamic ob- jects, and locally coherent motion while maintaining high reconstruction quality and computational efficiency. The first contribution addresses unbounded scene representation. Conventional NeRF- based methods often assume that the scene lies within a bounded volume, which makes it difficult to represent distant structures or large-scale environments. To overcome this limitation, this dissertation interprets the relationship between unbounded and bounded spaces from a projection perspective. Based on this interpretation, it intro- duces a p-norm-based mapping function that adaptively controls how unbounded space is embedded into a bounded representation domain. This mapping allows the represen- tation to allocate the bounded space more effectively according to scene geometry. The second contribution focuses on efficient dynamic scene representation using 3D Gaussian Splatting. Dynamic scenes require temporal modeling, but applying dy- namic representations to the entire scene introduces unnecessary cost because many regions remain static. To address this issue, this dissertation proposes a motion-based static-dynamic separation strategy. Static regions are represented with a compact static Gaussian representation, while dynamic regions are modeled with a keyframe-based in- terpolation scheme. This design enables efficient modeling of time-varying scenes while preserving the fast optimization and rendering advantages of explicit Gaussian repre- sentations. The third contribution improves the motion behavior of dynamic Gaussian repre- sentations. Although photometric reconstruction loss can produce plausible renderings, it does not explicitly constrain the underlying Gaussian trajectories. As a result, neigh- boring Gaussians may move independently even when they belong to the same local structure. To encourage locally coherent motion, this dissertation proposes ray-based Gaussian grouping and regularization terms for motion and shape consistency. The grouping identifies locally related Gaussians based on their rendering contributions, and the regularization encourages similar motion directions while preserving the local shape distribution over time. Through these contributions, this dissertation advances image-based 3D represen- tation for real-world novel view synthesis. By addressing unbounded space, dynamic scenes, and local motion coherence, the proposed methods improve the expressiveness and efficiency of 3D scene representations in practical settings.