Denoising Reconstructed Mesh under Single-view using Marching Cubes and Differentiable Rendering

Abstract

In this thesis, we propose a novel approach that denoises reconstructed mesh fused with only a single pair of RGB-D frames without using any off-the-shelf depth-filtering approaches, by exploiting mesh generation pipeline and given color image data. We first exploit core mechanism of mesh reconstruction pipeline, Marching Cubes[1], to generate multiple piecewise smooth meshes from only single depth image. Denoised depth is calculated by simply averaging depths from projection of generated meshes. Additionally, powered by recent advances of Differentiable Rendering technique, we propose a loss function that indicates geometric difference between captured color image and the rendering of input mesh, so that differentiable renderer can further denoise input mesh.