Calibration and 3D Pose Estimation of a Low-cost IMU and a Rolling Shutter Camera

Abstract

A rolling shutter camera and a 6 degree-of-freedom (DOF) inertial measurement unit (IMU) have received great attention over the past decade thanks to their low-cost sensing capability and complementary properties for localization of a mobile platform. However, a rolling shutter camera and a low-cost IMU have undesirable artifacts because of the line-by-line imaging of a rolling shutter camera and imperfect calibration of a low-cost IMU. In this thesis, we address 1) calibration and 2) relative/absolute pose estimation problems, which are the fundamental factors for visual-inertial simultaneous localization and mapping and structure-from-motion, considering the characteristics of a rolling shutter camera and a low-cost IMU.

For calibration of a rolling shutter camera and a low-cost IMU, we propose a new method that jointly estimates calibration and noise parameters of the low-cost IMU and the rolling shutter camera for effective sensor fusion in which accurate sensor calibration is very critical. Based on the graybox system identification, the proposed method estimates unknown noise density so that we can minimize calibration error and its covariance by using the unscented Kalman filter. Then, we refine the estimated calibration parameters with the estimated noise density in a batch manner. Experimental results on synthetic and real data demonstrate the accuracy and stability of the proposed method and show that the proposed method provides consistent results even with an unknown noise density of the IMU. Furthermore, experiments using a commercial smartphone validates the performance of the proposed calibration method in off-the-shelf devices.

For relative and absolute pose estimation, we exploit gyroscope measurements, angular velocity, along with image measurement to compute the relative and absolute pose of rolling shutter cameras. The gyroscope measurements provide information about instantaneous motion that causes the rolling shutter distortion. Having gyroscope measurements in one hand, we simplify the relative and absolute pose estimation problem, and then find a minimal solution for the problems. In order to solve the problem, we exploit the Gr¨obner basis and the direct linear transform methods for relative and absolute pose estimation problem, respectively. The proposed methods require only five points while the conventional methods with no gyroscope measurements demand 20 or 44 points for relative pose estimation and six points for absolute pose estimation. Moreover, the proposed methods are more efficient than the conventional methods owing to the smaller number of minimal points and solutions. Experimental results on synthetic and real data verify the superiority of the proposed method over conventional pose estimation methods.