Abstract
This short article describes the method of digital cytopathology using Z-stack scanning with or without extended focusing. This technology is suitable to observe such thick clusters as adenocarcinoma on cytologic specimens. Artificial intelligence (AI) has been applied to histological images, but its application on cytologic images is still limited. This article describes our attempt to apply AI technology to cytologic digital images. For molecular analysis, cytologic materials, such as smear, LBC, and cell blocks, have been successfully used for targeted single gene detection and multiplex gene analysis with next-generation sequencing. As a future perspective, the system can be connected to full automation by combining digital cytopathology with AI application to detect target cancer cells and to perform molecular analysis. The literature review is updated according to the subjects.
Introduction
Digital pathology has been used for primary histopathology diagnosis [1, 2]. And these scanned images are used for image analyses, which include histology, immunohistochemistry, and quantitative tissue morphometrics [3-5]. Digitalization of cytology images has been attempted, and now the digital images are successfully used for cytological diagnosis and various analyses [6-9]. Applications of digital cytopathology have been done with Z-stacks because not infrequently neoplastic cells form thick clusters that are difficult to scan with only 1 plane. Summation of these individual planes is called as “extended focusing” [9] and can be applied successfully to observe thickened cell clusters such as adenocarcinomas.
Applications of artificial intelligence (AI) technologies have been attempted mostly in digital histopathology, and its applications in digital cytopathology are very limited [10-14]. For molecular analysis, companion diagnostics, and next-generation sequencing, FFPE materials have been used as standard procedures. Recently, cytologic materials such as smears, LBC, and cell blocks have been successfully used for the above molecular analyses [15-19]. This article was aimed at demonstrating the application of AI technologies to digitized cytopathology images and to the utilization of selected tumor cells for molecular analysis.
Design and Methods
Digital Cytopathology and Application of AI Technologies
Cytologic smears of a total of 6 cases of pleural fluids with metastatic adenocarcinoma of the lung were scanned by NanoZoomer S210 (Hamamatsu Photonics KK, Shizuoka) with Z-stack focusing at 1 μm from −5 μm to +5 μm. The scanning time for samples 1–6 were 5 m 18 s, 16 m 28 s, 13 m 32 s, 14 m 52 s, 19 m 1 s, and 14 m 11 s (m: minutes, s: seconds), respectively.
Thick clusters of lung adenocarcinoma cells are well illustrated by Z-stack scanning. The images captured by NanoZoomer S210 are illustrated in Fig. 1. These individually focused images of different planes can be put together into 1 plane with “all levels are in focus”; this is designated as “extended focusing.” This function is saved in the scanner 3DHISTECH (Fig. 2). For the AI applications, Z-stack images of the 6 cases were used as illustrated (Fig. 3).
Z-stack view of adenocarcinoma clusters in the pleural fluid. a −5 μm. b 0 μm.
Z-stack images of adenocarcinoma aggregates in the pleural fluid. The adenocarcinoma cells in particular planes are in focus (a, b). Extended focusing (c) indicates that adenocarcinoma cells in all planes are in focus.
Z-stack images of adenocarcinoma aggregates in the pleural fluid. The adenocarcinoma cells in particular planes are in focus (a, b). Extended focusing (c) indicates that adenocarcinoma cells in all planes are in focus.
Application of the AI system to the adenocarcinoma cells in Z-stack ① on 1 plane. The carcinoma cells ② can be detected individually or in clusters ③. Papanicolaou stain. AI, artificial intelligence.
Application of the AI system to the adenocarcinoma cells in Z-stack ① on 1 plane. The carcinoma cells ② can be detected individually or in clusters ③. Papanicolaou stain. AI, artificial intelligence.
Image analysis was conducted by means of the technologies developed by NEC Corporation for digital pathology, including annotation schemes, feature extraction, and machine learning such as convolutional neural network [20, 21]. This system successfully detects the areas of gastric adenocarcinomas on tissue sections [20]. We applied these techniques to cytologic materials to detect the cancer cells. The analysis method for AI included the following. (1) Target cell clusters were selected by an experienced pathologist (a single cluster contains 20–30 cells in average). (2) A 20× image was cut out in a well-focused 0-level Z-stack layer of the Whole Slide Imaging. (3) For the extracted image, we estimated each tumor cell area by semantic segmentation (pixel-level classification). To estimate each tumor, we used U-Net with a 256 × 256-px input size (optimizer: Adam, learning rate 10−4, loss function: BCE Dice Loss; data augmentation: flip and color variations) [22, 23].
By applying these AI methods mentioned above, the system detected most of the clusters of adenocarcinoma on the captured images. The images were captured with ×20 magnification (Fig. 3). As an example, these technologies detected small clusters of adenocarcinoma of the lung in the pleural fluid as demonstrated (Fig. 4). Many lymphocytes, macrophages, and mesothelial cells were not labeled. The selected adenocarcinoma cells can be annotated on the images. Exceptionally, a few overlapping and darkly stained cells, probably histiocytes or mesothelial cells, were detected as suspicious for “positive” cells.
By applying the AI system to Pap-stained smears of the pleural fluid, various shapes and sizes of clusters of carcinoma cells can be detected (red: adenocarcinoma cells). Lymphocytes (a, d), histiocytes (b, e) and mesothelial cells (c, f) are not labeled. The cancer cells by Papanicolaou stain (a–c) are detected by AI application (d–f) (in red). AI, artificial intelligence.
By applying the AI system to Pap-stained smears of the pleural fluid, various shapes and sizes of clusters of carcinoma cells can be detected (red: adenocarcinoma cells). Lymphocytes (a, d), histiocytes (b, e) and mesothelial cells (c, f) are not labeled. The cancer cells by Papanicolaou stain (a–c) are detected by AI application (d–f) (in red). AI, artificial intelligence.
Molecular Cytopathology of Cytologic Materials
Using the same cytologic materials as those scanned with Z-stacks, selected clusters of adenocarcinoma cells were subjected to molecular analysis, for example, EGFR mutations, that is, point mutation and deletion. The selected and marked clusters of adenocarcinoma were microdissected (scraped off) manually (Fig. 5) after removing coverslips, and the following molecular analyses were done. In brief, the cells in microtubes were subjected to heat-block at 75° and proteinase K treatment for >1 h, and subsequently DNA was extracted by Maxwell® 16 AS2000 (Promega). We used SNP-based mutation detection using i-densy® (Arkray, Japan) [25, 26]. This equipment can detect 3 mutations at a time, such as EGFR exon 21 L858R, exon 19 deletion, and exon 20 T790M, on the same specimen.
Workflow from removal of coverslips (a), to microdissection (b) and submission to i-densy® to analyze EGFR mutation (c) after extraction of DNA from cancer cell clusters. Mutational analysis demonstrates 2 different point mutations (EGFR mutation exon 19 del and exon 20 T790M) in the circled adenocarcinoma cells.
Workflow from removal of coverslips (a), to microdissection (b) and submission to i-densy® to analyze EGFR mutation (c) after extraction of DNA from cancer cell clusters. Mutational analysis demonstrates 2 different point mutations (EGFR mutation exon 19 del and exon 20 T790M) in the circled adenocarcinoma cells.
The microdissected carcinoma clusters were analyzed by i-densy®, and the mutations in EGFR gene were successfully detected (Fig. 5). By using this method, minimally required numbers of carcinoma cells were calculated as 30 (unpublished data).
Discussion and Conclusion
Computational and Molecular Cytopathology: Current and Future Perspectives
Our results showed promising results for cytopathology by applying a previously established AI system on histologic sections [20]. This system could be applied for the limited target of the cells, which are adenocarcinomas of the lung in the pleural fluid. The system detects the atypical cells, most of which are adenocarcinoma cells, individually or in clusters. But some overlapping non-neoplastic cells or more darkly stained cells such as histiocytes or mesothelial cells, respectively, were detected as suspicious for malignant cells. This system requires more cases with deep learning and machine learning in order to improve its detection accuracy. For molecular pathology detection, these marked (annotated) cell clusters were microdissected manually and could be used to detect specific genomic changes, mutations, and deletions. Semi-automated laser capture microdissection of cytology samples such as adenocarcinoma of the lung has been reported [27]. It is expected to apply this technology to microdissect the target cancer cells for molecular analysis.
When the AI system and microdissection of the selected target cells could be unified, the fully automated system from the scanning of cytologic specimens to the molecular analysis could be accomplished. It is expected that this fully automated system would be a very efficient system, particularly when the targeted cancer cells are extremely small in number.
Acknowledgements
The authors wish to thank Hamamatsu Photonics, Shizuoka, and e-path Ltd. Co. for their excellent technical support in digitization of cytologic specimens. Part of this study was presented at the 20th International Congress of Cytology (ICC) in Sydney 2019.
Statement of Ethics
This study is the collaboration between Nippon Koukan Hospital and NEC Corp. and has been approved by the Ethical Committees at Nippon Koukan Hospital.
Conflict of Interest Statement
The authors do not have any conflict of interest statement to declare regarding this study.
Funding Sources
The authors have received no funding to pursue this study.
Author Contributions
Robert Y. Osamura: digital cytology and molecular cytology. Naruaki Matsui: micro- and macrodissection of target cancer cells. Masato Kawashima: molecular cytology and technical support. Hiroyasu Saiga: digital cytology and AI application. Maki Ogura: digital cytology and AI application. Tomoharu Kiyuna: digital cytology and AI application.