Multi-Task Deep Neural Network for Instance Segmentation and Ordering Recovery

Abstract

The field of computer vision has witnessed remarkable advancements, extending beyond mere object detection and segmentation in images. Current research endeavors focus on comprehending the relationship between objects, thereby capturing their high-level context. In particular, many studies are being conducted to identify the occlusion order relationship between objects in an image. However, existing studies predominantly rely on accurately annotated object masks to identify the occlusion order relationships. In practical scenarios, obtaining precise object masks is often challenging, necessitating the need to infer occlusion order relationships from predicted masks. As a result, we propose an end-to-end network that leverages object masks from input images and subsequently extracts occlusion order relationship from the predicted masks. To provide comprehensive understanding of our proposed approach, the thesis is structured as follows. Chapter 1 serves as an introduction, outlining the purpose of our study and delving into the existing body of research relevant to occlusion order relationship identification. In Chapter 2, we introduce the existing studies on recoverin occlusion order relationships. It takes ground truth masks as an input and recovers occlusion order relationships. A matching algorithm between predicted masks and ground truth masks is introduced in Chapter 3. This method makes it possible to determine the occluding order relationship between objects even when the predicted masks does not match the ground truth masks in shape and number. Using this matching algorithm, we propose an end-to-end network that learns instance segmentation and occlusion order relationships at once. Performance comparison between the method in Chapter2 and the proposed method is presented. Finally, Chapter 4 provides a comprehensive conclusion, summarizing the key findings and contributions of our study. We expect this result to work well even in scenarios where accurate object masks are unavailable.