A Study of Methods for Accurate Depth Map Estimation using Techniques of Computational Camera and Photography

Abstract

From the beginning of 21st century, computational photography is evolving rapidly with advances in electronic sensing, processing and storage. Computational photography includes the methods of processing multiple images from a traditional camera, to acquire information which overcome the limitations of the camera and produce a new type of image. In particular, the restoration of the shape of the object is a major research topic in this field, various studies have been conducted. One of these studies, Shape from Focus (SFF), can obtain a fast and accurate reconstruction result by obtaining the point where the focus operator’s response is greatest from several images having different depths of field. However, in order to acquire an image, hardware facilities other than a camera are essential. In the process of acquiring images, problems such as hardware micro-vibration, electronic error, control error, measurement time, and expensive equipment make measurement difficult. Various SFF methods have been developed to solve this problem, but the traditional methods have limitations in restoration accuracy. In this paper, we proposed a new depth map measurement method using the new SFF and light field (LP) photography to solve this problem. The images with different depths can be obtained by changing the distance between the lens and the image sensor or by changing the distance between the object and the camera lens along the optical axis at regular intervals. In this process, mechanical vibrations are generated by the mechanical elements of the system equipment, and as a result, the object cannot be captured at regular intervals, thereby making it impossible to estimate the exact shape of the object. Traditional SFF methods do not take this vibration into account, limiting the accuracy of the reconstructed shape. In this study, we used an adaptive neural network (ANN) filter. The mechanical vibration noise is modeled using Gaussian and non-Gaussian distribution, considering the actual situation. The ANN filter is designed as a preprocessing filter to find the original position of each frame of the input image. As a result of applying the proposed method to the image of synthetic and real objects, we can see the improved performance of the depth estimation compared to the existing methods. Typically, the SFF system incurs significant errors from images, including noise. As a solution to this problem, a simple low pass filter such as the Gaussian filter has generally been used in most studies. However, when a low filter is applied, the noise is depressed but the signals are also blurred, which results in inaccuracies regarding the depth map. In this study, the noise is depressed independently without losing the original signals, and the edge components, which play important roles in finding a focused surface, are enhanced using the independent component analysis (ICA). The edge signals are amplified with a simple basis vector correction in the IC vector space. The experiments are implemented with simulated objects and real objects. The experimental results demonstrate that the obtained accuracy is comparable to that of existing methods. Finally, to reduce the cost of the system and to investigate the applicability of SFF techniques to light field data, the focused image surface (FIS) searching using the LF camera was studied. FIS of an object is defined as the surface formed by the set of points at which the object points are focused by a camera lens. Traditional SFF methods treat the scene as a plane parallel to the focal plane (assuming an equifocal). These assumptions is not vaild due to complex shapes of objects in most practical scenes, and as a recult, conventional methods have limitations on the accuracy of the reconstructed shape. This limitation can be improved by introducing a focused image surface (FIS). However, existing methods for doing FIS search are expensive. In addition, the image data generated from the LF data generates aliasing artifacts, and as a result, the accuracy of shape restoration is degraded. In this study, we use a focus operator that handles aliasing artifacts and apply superpixel algorithm to reduce the computational cost. In order to evaluate the performance of the restoration results, we compared the existing methods in terms of time and speed. As a result of applying the proposed method to the images of synthetic and real objects, we can see the performance improvement compared to the existing methods.