Depth map Super-resolution using RGB Guidance

Abstract

In the recent, many smartphones have various cameras such as the Time-of-Flight(ToF) camera, depth sensor and depth camera. It helps to get a depth map and depth information. Depth maps are an image that displays camera-to-object distance data. Deep learning-based depth map super-resolution technology is required to solve this problem because it is expensive to obtain a high-resolution depth map. The proposed method generates a high-resolution depth map using a low-resolution depth map through a three-step deep learning model based on SRCNN, which is proposed as a deep learning based on super-resolution. Through the use of the skip-connection, we improved the efficiency of learning by using the difference between input and output, and by applying the L1 loss function instead of the L2 loss function. And we use the Leaky ReLU activation function instead of ReLU function. This thesis method solves this problem by using Leaky and ELU (Exponential ReLU) method to solve the problem of dying ReLU which is a problem of ReLU which is well known as the activation function. And we used Adam optimizer instead of SGD or Momentum optimizer. The proposed method is compared with the conventional Li et al method, we obtained a significant improvement of 7~24% for the scale factors 4x, 8x and 16x in NYUDv2, SUN RGBD and Middlebury, three famous depth map datasets. However, the experimental results of raw depth value in the NYUDv2 dataset showed improved results at scale factor 4x and 8x, but not at scale factor 16x.