Geometrical Characteristics Estimation using Unsupervised Learning Beyond the Light Field Images

Abstract

In order to exactly express the characteristics of real-world throughout the computer vision technique, high level scene capturing device and various sensors are required. Nowadays, light field cameras have become popular capturing devices which can be easily available in normal consumer stocks. The main advantage of light field camera is that multi viewpoint scenes and various angular directional scenes are obtained by single light field camera due to the microlens array. Since the light field scenes are composed of spatial and angular domain, a depth map can be estimated using passive depth estimation approaches. In addition, as the computational power is increase due to the graphics processing unit (GPU), deep learning approaches are widely applied in various computer vision applications such as monocular depth estimation, object tracking and motion estimation. In this thesis, we mainly deal with the exploiting geometrical characteristics of 3-dimensional properties to change the 2-dimensional light field scene into the 3-dimensional applications with deep learning techniques. The captured light field scene from microlens array has small spatial and angular resolution compare to the captured by stereo camera system. Even though the estimated depth map from light field scene shows high accuracy, it is not sufficient when it compared with active sensor based depth map. Moreover, conventional depth estimation algorithms do not significantly consider occlusion and discontinuity regions while estimating a depth map. To address that kinds of issues, we propose the novel learning based light field depth estimation method. In addition, beyond the light field images, we trying to extend learning technique to sequential stereo scenes. Due to abruptly increased requirement of human-free autonomous driving technique, we invented geometrical characteristics estimation method from moving scenes such as optical flow and camera ego-motion as well as depth map. Our knowledge of this thesis is mainly divided into two subjects: 1) spatial-angular super resolution based depth estimation for light field scenes. 2) depth, camera ego-motion and optical flow estimation from sequential stereo scenes. In the first subject, we enlarge the spatial and angular resolution of light field image, to compensate limitations of the low resolution problem which occurred by properties of light field camera. Through the up-scaled light field image, we generate a depth map by using a light field epipolar plane image and style transfer network which executed on unsupervised learning scheme. At the second subject, the extended application of unsupervised learning technique is applied to estimate depth map, ego-motion and optical flow from stereo sequence. To improve the accuracy of various factors, we defined coupled consistency conditions in our network architecture. The flow consistency and synthesis consistency help us induce various factors through the unsupervised scheme. We will conclude the thesis with extensive experiments for two subjects and several promising technique for unsupervised learning based geometrical scene analysis.