Using Pseudo-Supervision for Improving Model Generalization

Abstract

In recent years, significant performance improvement on the various computer vision tasks has been made, facilitated by the advances on deep learning methods and massively annotated data. Particularly, it is established as a de-facto standards in research area that the model performance increases as the amount of the data increases. However, creating annotated dataset for training is costly and time-consuming, which is arguably the major hurdle to the growth of dataset scale. For example, datasets with automatically-generated annotations could be two magnitudes larger than densely-annotated ones in terms of total duration. Considering this, the self-training approach using pseudo-supervision has received tremendous attention from research society because of its usefulness to exploit the unlabeled dataset. Because of simplicity and effectiveness of it, many researchers have adopted it for various applications. In this paradigm, a pre-trained model first generate pseudo labels on the unlabeled dataset, and then the model for target task is trained by using it. In this thesis, we focus on the importance of pseudo-supervision for addressing multi- modal computer vision tasks, which can be a significant role for model generalization. Specifically, we generate the pseudo-supervision using two knowledge sources (i.e., external and internal knowledge) for improving the performance of two different tasks (i.e., natural language video localization and story visualization), respectively.