Deep learning based cancer detection from whole slide images integrating contextual information

Abstract

The efficiency of the histopathological image analysis by deep learning typically requires precise annotation, for which the regions of suspicion need to be scrupulously demarcated by the pathologists. Exhaustive annotation of the giga-pixel-sized whole-slide images (WSIs) is highly time-consuming and labor-intensive. Typically, computational pathology routinely considers the tumor microenvironment while analyzing WSIs to improve diagnostic accuracy and precision. In this regard, the current research aimed to demonstrate that a novel deep neural network, ContXNet, which we developed, considers such contextual features in WSIs and can accurately detect cancer regions even with the roughly annotated WSI data. For this purpose, we first designed a neural-network model based on self-attention techniques to learn features of the heterogeneous tumor microenvironment from the multiscale image patches obtained from 16 weakly annotated WSIs of colorectal cancer specimens from Asan Medical Center. The model achieved accuracy, precision, recall, and F1 scores of 97.18%, 100%, 97.16%, and 98.56%, respectively, on an unseen dataset of 24k cancer patches. Deep neural networks incorporating the contextual histopathological features in WSIs are hypothesized to enhance the performance in diagnostic and prognostic tasks for computer-aided pathology. Experimental analyses conducted on Patchcamelyon and BreakHis datasets revealed that the proposed deep learning model, to the best of our knowledge, achieved a state-of-art performance on both datasets, e.g., with accuracy and AUC score of 99.03% and 0.9993, respectively, on the Patchcamelyon dataset. In the analysis of the pseudo-H&E data acquired by the newly developed oblique plane microscopy (OPM), the proposed ContXNet method performed with an accuracy significantly higher than those found in other convolutional neural network (CNN) models.