Multimodal deep learning based assessment of meibomian glands with segmentation and adversarial enhancement

Abstract

Meibomian gland dysfunction (MGD) is one of the common problems for dry eye disease, and based on different countries, up to two-thirds of the elderly population worldwide suffers from MGD. Yet, when assessing the MGD by ophthalmologists, the consistency for evaluating the Meibomian gland (MG) is very low, resulting in an inaccurate diagnosis. So, it is highly imperative to develop an effective, robust, and automated MG assessment system for early diagnosis of dry eye disease. Despite advances in deep learning, its application in MG assessment is limited by a lack of abundant images and ground truth labels. In this study, we propose a multimodal deep learning approach to segment gland and eyelid information with subpixel accuracy to visualize MG of variable contrast with a neural network. Experimental results empirically demonstrate that our deep learning methods outperform existing manual segmentation results of ophthalmologists with 67.63% and 94.03% accuracy in the case of glands and eyelid segmentation, respectively. A residual network model was designed to provide an assessment of MGD. Knowledge transferred from the predicted glands and eyelid mask combined with original images achieve assessment accuracy of 70.38%, overcoming the manual assessment score of 65.06%. Furthermore, transfer learning of a generative adversarial network (GAN) was used to remove saturated glares from direct reflection from the original images. We are also publishing a dataset of MG online containing 3000 images. Every 1000 infrared images are paired with precise segmentation masks of glands, eyelid, and six graded meiboscore results by two ophthalmologists with metadata to allow further research and advancements in this field.