SmartFace: An Intelligent Device for Security and Surveillance

Abstract

Security and surveillance monitoring are essential elements of public safety. Digital video cameras are used in various settings such as government offices, schools, living apartments, corporate offices, and industry surveillance to monitor video feeds. However, storing, analyzing, and verifying an individual's identity in the camera footage can be challenging due to the limited cognitive power of the human brain. Additionally, the exponential amount of data generated by each camera requires large dedicated servers, GPU machines, infrastructure, and human resources to analyze live streams.

This thesis introduces face recognition applications for security and surveillance. However, directly using such large-scale video data for face recognition models is challenging due to the need for additional data preprocessing. The annotation of generated image frames from video cameras requires many annotators and is a complex and time-consuming task. Furthermore, the existing face recognition datasets harvested from the internet contain noisy images, labels, and ambiguous images, which can reduce face recognition accuracy when used to train convolutional neural network models.

This thesis addresses the issues associated with large-scale face recognition datasets and proposes learning methods to resolve uncertainties in such datasets. To alleviate the carbon emissions associated with GPU processing and infrastructure costs, this thesis presents an implementation of an on-device smart-face system and introduces its potential application for banking authentication systems. Additionally, this thesis introduces generalized face recognition using contrastive learning and an extension to federated learning to achieve decentralized training by preserving internal device data. The proposed approach is evaluated on a collected dataset and public benchmark, demonstrating its effectiveness.