Image Reconstruction Techniques Using Auxiliary Features

Abstract

Image reconstruction is a fundamental problem, and its goal is to estimate an unknown ground truth image from an observed noisy image. Image reconstruction methods rely on a only sole input image, leading to challenges in scenarios where the input is corrupted by serve noise. However, auxiliary features can guide the image reconstruction methods to preserve image details because those include fine details with much less noise. In this thesis, we propose three image reconstruction methods to exploit auxiliary features effectively. We first propose a multi-view image denoising method that can find similar image patches in multiple images so that image denoiser can reduce noise more effectively by utilizing the recognized similar patches from multi-view images. We embed all image patches in multi-view images into a low-dimensional space, and it facilitates the image denoiser to identify similar image patches effectively within the space. Secondly, we propose a deep learning-based image reconstruction method that takes a flash/no-flash image pair as input and generates a denoised no-flash image while robustly handling inconsistencies between the input image pair. Our reconstruction method infers a consistent flash image patch, which is structurally similar to the ground truth, and combines the no-flash image and inferred consistent flash image. Lastly, we present a new reconstruction method for Monte Carlo rendering. Our reconstruction method is based on a transformer with a new self-attention mechanism that extracts dual attention scores from a noisy color image and auxiliary features so that we can robustly infer a joint similarity from the two input sources with different characteristics. In this thesis, we demonstrate that our methods improve the reconstruction performance more than state-of-the-art methods in various test cases.