Classification of staining patterns on HEp-2 specimen images (PhD)

Abstract

Indirect immunofluorescence (IIF) imaging of human epithelium type-2 (HEp-2) cell substrate has been considered as the ‘gold standard’ test for diagnosis of autoimmune disorders. However, manual diagnosis assessment involves tedious and labor-intensive workflow, and can also result in with high intra-personnel and intra-laboratory variations. In this thesis, an image-processing based algorithmic framework is presented for identification of different staining patterns manifested on HEp-2 cell substrate, which can contribute to the computerassisted diagnosis of autoimmune disorders. The visual staining patterns can be broadly divided into mitotic patterns and interphase patterns, based on their manifestation during different phases of the cell-cycle. Owing to the fact that mitosis is a transition phase, the appearance of patterns associated with the mitotic phase is quite rare, which makes the identification of mitotic patterns more challenging and important. Another challenge considered in this work includes designing the framework using only primary channel images, without the use of DAPI pattern images. The DAPI is a secondary chemical dye, which is required for the acquisition of segmentation masks for cells, but not recommended in practice. In addition, less inter-class and high intra-class variations among samples make the classification problem more difficult to solve. Considering above mentioned challenges, the contributions of the thesis include novel cell-based, region-based, and specimen-based approaches for identification of patterns on specimens. The cell-based approaches are designed to identify the rare or minority based patterns, as the traditional classification paradigm gets biased towards majority class samples in specimen-based approaches. Hence, minority patterns can be identified via cell-level processing. These approaches use segmentation mask images for cell segmentation and cell extraction from specimen images. In cell-based approaches, the first proposed framework isthe identification of rare mitotic phase cells in specimen images. Importantly, this considers the data imbalance between mitotic patterns and non-mitotic/interphase patterns in classification paradigm, and we use different data-skew balancing strategies. The next framework focuses on mitigating some drawbacks of data-skew balancing strategies, and considers mitotic pattern cells as anomaly, in the specimen images, that consist of largely interphase pattern cells and few mitotic pattern cells. This task is done by one class classifier, used as an anomaly detection framework. The next approach involves a similarity-learning based framework, for identification of the mitotic patterns in specimen images. In this approach, a distance metric computation is integrated with deep convolutional neural networks (DCNN), aiming to learn useful embeddings of the data, using triplet-loss based function. All the cell-based strategies also involve a novel and separate decision-making criterion for mitotic specimen, that is based on the count of a minimum number of mitotic samples, present in the specimens. This criterion is termed as ‘threshold-based decision-making criterion’. Following the proposed cell-based approaches, the next task of this thesis work is to address the identification of mitotic/minority patterns in specimen images, via image-based approaches, in order to avoid the use of the provided segmentation masks, which require the DAPI dye. This problem statement is also addressed in three different ways, where the first idea is to perform classification via a segmentation approach. Though, this pipeline is also based on a cell-based approach, the segmentation masks are computer automatically. Here, a U-Net neural network-based segmentation framework is used, followed by a 2-class classification framework. The second approach divides specimen images in local uniform regions, where these regions are considered as input images to a CNN-based classifier. The third and final method considers mitotic patterns as distinct objects in specimen images, due to their rare appearance, and a good object-detection framework is applied for detection of such patterns. Hence, these three approaches address the identification of minority patterns in complete specimen images. Following the various strategies for identification of mitotic cells, as a final task of the thesis, a framework is designed for the identification and classification of different interphase type patterns, which can be discriminated based on their morphology and appearance. Hence, considering the characteristic of each class, some class-specific feature representations are designed to capture such types of discriminative characteristics of each pattern. Similar to the mitotic class, here also, D-CNN based framework is used for the identification of different interphase patterns. Finally, another novel attempt is made, in order to generate synthetic samples for rare classes, so that the synthetic samples can contribute to the training of the classifier, where the data across classes was imbalanced in the original scenario. For this task, a generative adversarial network-based framework is used for mitotic patterns, which is validated experimentally and subjectively by medical experts. In experimental results and analysis, it is demonstrated that the proposed frameworks perform quite well for the identification of mitotic patterns, as well as for the classification of different interphase patterns. Overall, the best performance (in terms of Matthews correlation coefficient) for mitotic class (for single cell images) is 0.99. For specimen images, the best achieved performance for mitotic class, and interphase classes are 0.96 and 0.98, respectively. The study also shows a comparative analysis amongst different proposed frameworks and with existing works. Thus, the proposed methods are validated, with good and notable performance, for identification of different staining patterns, on an important problem definition.