Hi-Fidelity Three Dimensional Projection Mapping onto non-rigid Objects

Abstract

In this thesis, we introduce a new method to achieve hi-fidelity projection mappings onto real world three dimensional and non-rigid objects. Projection mapping is a technique that allows us to project images onto an object according to its position, shape and size. Such projection of images is called Spatial Augmented Reality (SAR). In order to achieve a hi-fidelity projection mapping, photometric and geometric aspects of the target object (projection surface) must be taken into consideration. In this thesis we propose methods to enhance visual similarity of a projection mapped object when compared to a real object. A lot of research has been done on fully overlapping projections for 3D projection mapping systems but reproducing a high quality appearance using this technology still remains a challenge. On top of existing color compensation based methods, much effort is still required to faithfully reproduce an appearance that is free from artifacts, colorimetric n consistencies and inappropriate illuminance over a three dimensional projection surface. According to our observation, this is due to the fact that overlapping projections are treated as an additive-linear mixture of color. However, this is not the case according to our elaborated observations. In this thesis, we propose a novel method that enables us to use high quality measured data from original objects and regenerate the same appearance by projecting optimized images onto similarly shaped mockups using multiple projectors that ensure projection-rendered results look visually close to the real object. Next, geometric accuracy of the projection mapping is also of prime importance. In this thesis we present a method that ensures that a high accuracy object-texture registration. Fiducial based tracking of deformable non-rigid surfaces for projection mapping purposes has been widely used but efficient handling and representation of fine deformations and sharp contours in such surfaces still needs to be addressed. On top of existing fiducial based projection mapping methods, a lot of work is required to achieve a projection mapping that is free from geometric errors, misalignments, and artifacts in the areas that contain such deformations. This is due to the fact that existing methods that rely on fiducial markers have used markers that sparsely represent the projection surface, are large in size, are not sub-divisible and projector-camera registration is not accurate enough. Here, we propose an adaptive method that enables us to accurately track a deformable non-rigid surface and project textures onto it, ensuring that no visible misalignments occur. We have demonstrated our results using a sheet of paper and a cloth under several geometric deformations. Moreover, to demonstrate high fidelity color reproduction, we have used celadons from Goryeo dynasty. Our experimental results show compelling results in terms of visual similarity, colorimetric and geometric error.