This short article describes the method of digital cytopathology using Z-stack scanning with or without extended focusing. This technology is suitable to observe such thick clusters as adenocarcinoma on cytologic specimens. Artificial intelligence (AI) has been applied to histological images, but its application on cytologic images is still limited. This article describes our attempt to apply AI technology to cytologic digital images. For molecular analysis, cytologic materials, such as smear, LBC, and cell blocks, have been successfully used for targeted single gene detection and multiplex gene analysis with next-generation sequencing. As a future perspective, the system can be connected to full automation by combining digital cytopathology with AI application to detect target cancer cells and to perform molecular analysis. The literature review is updated according to the subjects.

Digital pathology has been used for primary histopathology diagnosis [1, 2]. And these scanned images are used for image analyses, which include histology, immunohistochemistry, and quantitative tissue morphometrics [3-5]. Digitalization of cytology images has been attempted, and now the digital images are successfully used for cytological diagnosis and various analyses [6-9]. Applications of digital cytopathology have been done with Z-stacks because not infrequently neoplastic cells form thick clusters that are difficult to scan with only 1 plane. Summation of these individual planes is called as “extended focusing” [9] and can be applied successfully to observe thickened cell clusters such as adenocarcinomas.

Applications of artificial intelligence (AI) technologies have been attempted mostly in digital histopathology, and its applications in digital cytopathology are very limited [10-14]. For molecular analysis, companion diagnostics, and next-generation sequencing, FFPE materials have been used as standard procedures. Recently, cytologic materials such as smears, LBC, and cell blocks have been successfully used for the above molecular analyses [15-19]. This article was aimed at demonstrating the application of AI technologies to digitized cytopathology images and to the utilization of selected tumor cells for molecular analysis.

Digital Cytopathology and Application of AI Technologies

Cytologic smears of a total of 6 cases of pleural fluids with metastatic adenocarcinoma of the lung were scanned by NanoZoomer S210 (Hamamatsu Photonics KK, Shizuoka) with Z-stack focusing at 1 μm from −5 μm to +5 μm. The scanning time for samples 1–6 were 5 m 18 s, 16 m 28 s, 13 m 32 s, 14 m 52 s, 19 m 1 s, and 14 m 11 s (m: minutes, s: seconds), respectively.

Thick clusters of lung adenocarcinoma cells are well illustrated by Z-stack scanning. The images captured by NanoZoomer S210 are illustrated in Fig. 1. These individually focused images of different planes can be put together into 1 plane with “all levels are in focus”; this is designated as “extended focusing.” This function is saved in the scanner 3DHISTECH (Fig. 2). For the AI applications, Z-stack images of the 6 cases were used as illustrated (Fig. 3).

Fig. 1.

Z-stack view of adenocarcinoma clusters in the pleural fluid. a −5 μm. b 0 μm.

Fig. 1.

Z-stack view of adenocarcinoma clusters in the pleural fluid. a −5 μm. b 0 μm.

Close modal
Fig. 2.

Z-stack images of adenocarcinoma aggregates in the pleural fluid. The adenocarcinoma cells in particular planes are in focus (a, b). Extended focusing (c) indicates that adenocarcinoma cells in all planes are in focus.

Fig. 2.

Z-stack images of adenocarcinoma aggregates in the pleural fluid. The adenocarcinoma cells in particular planes are in focus (a, b). Extended focusing (c) indicates that adenocarcinoma cells in all planes are in focus.

Close modal
Fig. 3.

Application of the AI system to the adenocarcinoma cells in Z-stack ① on 1 plane. The carcinoma cells ② can be detected individually or in clusters ③. Papanicolaou stain. AI, artificial intelligence.

Fig. 3.

Application of the AI system to the adenocarcinoma cells in Z-stack ① on 1 plane. The carcinoma cells ② can be detected individually or in clusters ③. Papanicolaou stain. AI, artificial intelligence.

Close modal

Image analysis was conducted by means of the technologies developed by NEC Corporation for digital pathology, including annotation schemes, feature extraction, and machine learning such as convolutional neural network [20, 21]. This system successfully detects the areas of gastric adenocarcinomas on tissue sections [20]. We applied these techniques to cytologic materials to detect the cancer cells. The analysis method for AI included the following. (1) Target cell clusters were selected by an experienced pathologist (a single cluster contains 20–30 cells in average). (2) A 20× image was cut out in a well-focused 0-level Z-stack layer of the Whole Slide Imaging. (3) For the extracted image, we estimated each tumor cell area by semantic segmentation (pixel-level classification). To estimate each tumor, we used U-Net with a 256 × 256-px input size (optimizer: Adam, learning rate 10−4, loss function: BCE Dice Loss; data augmentation: flip and color variations) [22, 23].

By applying these AI methods mentioned above, the system detected most of the clusters of adenocarcinoma on the captured images. The images were captured with ×20 magnification (Fig. 3). As an example, these technologies detected small clusters of adenocarcinoma of the lung in the pleural fluid as demonstrated (Fig. 4). Many lymphocytes, macrophages, and mesothelial cells were not labeled. The selected adenocarcinoma cells can be annotated on the images. Exceptionally, a few overlapping and darkly stained cells, probably histiocytes or mesothelial cells, were detected as suspicious for “positive” cells.

Fig. 4.

By applying the AI system to Pap-stained smears of the pleural fluid, various shapes and sizes of clusters of carcinoma cells can be detected (red: adenocarcinoma cells). Lymphocytes (a, d), histiocytes (b, e) and mesothelial cells (c, f) are not labeled. The cancer cells by Papanicolaou stain (a–c) are detected by AI application (d–f) (in red). AI, artificial intelligence.

Fig. 4.

By applying the AI system to Pap-stained smears of the pleural fluid, various shapes and sizes of clusters of carcinoma cells can be detected (red: adenocarcinoma cells). Lymphocytes (a, d), histiocytes (b, e) and mesothelial cells (c, f) are not labeled. The cancer cells by Papanicolaou stain (a–c) are detected by AI application (d–f) (in red). AI, artificial intelligence.

Close modal

A recent literature review on the AI in cytopathology showed the applications of AI on thyroid, urine, pancreaticobiliary, and lung specimens [24]. For the pleural fluid, the detection rate of carcinoma cells by AI was very high, that is, sensitivity and specificity, 100% respectively [23].

Molecular Cytopathology of Cytologic Materials

Using the same cytologic materials as those scanned with Z-stacks, selected clusters of adenocarcinoma cells were subjected to molecular analysis, for example, EGFR mutations, that is, point mutation and deletion. The selected and marked clusters of adenocarcinoma were microdissected (scraped off) manually (Fig. 5) after removing coverslips, and the following molecular analyses were done. In brief, the cells in microtubes were subjected to heat-block at 75° and proteinase K treatment for >1 h, and subsequently DNA was extracted by Maxwell® 16 AS2000 (Promega). We used SNP-based mutation detection using i-densy® (Arkray, Japan) [25, 26]. This equipment can detect 3 mutations at a time, such as EGFR exon 21 L858R, exon 19 deletion, and exon 20 T790M, on the same specimen.

Fig. 5.

Workflow from removal of coverslips (a), to microdissection (b) and submission to i-densy® to analyze EGFR mutation (c) after extraction of DNA from cancer cell clusters. Mutational analysis demonstrates 2 different point mutations (EGFR mutation exon 19 del and exon 20 T790M) in the circled adenocarcinoma cells.

Fig. 5.

Workflow from removal of coverslips (a), to microdissection (b) and submission to i-densy® to analyze EGFR mutation (c) after extraction of DNA from cancer cell clusters. Mutational analysis demonstrates 2 different point mutations (EGFR mutation exon 19 del and exon 20 T790M) in the circled adenocarcinoma cells.

Close modal

The microdissected carcinoma clusters were analyzed by i-densy®, and the mutations in EGFR gene were successfully detected (Fig. 5). By using this method, minimally required numbers of carcinoma cells were calculated as 30 (unpublished data).

Computational and Molecular Cytopathology: Current and Future Perspectives

Our results showed promising results for cytopathology by applying a previously established AI system on histologic sections [20]. This system could be applied for the limited target of the cells, which are adenocarcinomas of the lung in the pleural fluid. The system detects the atypical cells, most of which are adenocarcinoma cells, individually or in clusters. But some overlapping non-neoplastic cells or more darkly stained cells such as histiocytes or mesothelial cells, respectively, were detected as suspicious for malignant cells. This system requires more cases with deep learning and machine learning in order to improve its detection accuracy. For molecular pathology detection, these marked (annotated) cell clusters were microdissected manually and could be used to detect specific genomic changes, mutations, and deletions. Semi-automated laser capture microdissection of cytology samples such as adenocarcinoma of the lung has been reported [27]. It is expected to apply this technology to microdissect the target cancer cells for molecular analysis.

When the AI system and microdissection of the selected target cells could be unified, the fully automated system from the scanning of cytologic specimens to the molecular analysis could be accomplished. It is expected that this fully automated system would be a very efficient system, particularly when the targeted cancer cells are extremely small in number.

The authors wish to thank Hamamatsu Photonics, Shizuoka, and e-path Ltd. Co. for their excellent technical support in digitization of cytologic specimens. Part of this study was presented at the 20th International Congress of Cytology (ICC) in Sydney 2019.

This study is the collaboration between Nippon Koukan Hospital and NEC Corp. and has been approved by the Ethical Committees at Nippon Koukan Hospital.

The authors do not have any conflict of interest statement to declare regarding this study.

The authors have received no funding to pursue this study.

Robert Y. Osamura: digital cytology and molecular cytology. Naruaki Matsui: micro- and macrodissection of target cancer cells. Masato Kawashima: molecular cytology and technical support. Hiroyasu Saiga: digital cytology and AI application. Maki Ogura: digital cytology and AI application. Tomoharu Kiyuna: digital cytology and AI application.

1.
Abels
E
,
Pantanowitz
L
,
Aeffner
F
,
Zarella
MD
,
van der Laak
J
,
Bui
MM
,
Computational pathology definitions, best practices, and recommendations for regulatory guidance: a white paper from the digital pathology association
.
J Pathol
.
2019 Nov
;
249
(
3
):
286
94
. .
2.
Zarella
MD
,
Bowman
D
,
Aeffner
F
,
Farahani
N
,
Xthona
A
,
Absar
SF
,
A practical guide to whole slide imaging: a white paper from the digital pathology association
.
Arch Pathol Lab Med
.
2019 Feb
;
143
(
2
):
222
34
. .
3.
Madabhushi
A
,
Lee
G
.
Image analysis and machine learning in digital pathology: challenges and opportunities
.
Med Image Anal
.
2016 Oct
;
33
:
170
5
. .
4.
Aeffner
F
,
Zarella
MD
,
Buchbinder
N
,
Bui
MM
,
Goodman
MR
,
Hartman
DJ
,
Introduction to digital image analysis in whole-slide imaging: a white paper from the digital pathology association
.
J Pathol Inform
.
2019
;
10
:
9
. .
5.
Pell
R
,
Oien
K
,
Robinson
M
,
Pitman
H
,
Rajpoot
N
,
Rittscher
J
,
The use of digital pathology and image analysis in clinical trials
.
J Pathol Clin Res
.
2019 Apr
;
5
(
2
):
81
90
. .
6.
Van Es
SL
,
Kumar
RK
,
Pryor
WM
,
Salisbury
EL
,
Velan
GM
.
Cytopathology whole slide images and adaptive tutorials for senior medical students: a randomized crossover trial
.
Diagn Pathol
.
2016 Jan 8
;
11
:
1
. .
7.
Wilbur
DC
.
Digital pathology and its role in cytology education
.
Cytopathology
.
2016 Oct
;
27
(
5
):
325
30
. .
8.
Hanna
MG
,
Pantanowitz
L
.
Reply to why is digital pathology in cytopathology lagging behind surgical pathology?
Cancer
.
2017 Jul
;
125
(
9
):
732
20
. .
9.
Capitanio
A
,
Dina
RE
,
Treanor
D
.
Digital cytology: a short review of technical and methodological approaches and applications
.
Cytopathology
.
2018 Aug
;
29
(
4
):
317
25
. .
10.
Tizhoosh
HR
,
Pantanowitz
L
.
Artificial intelligence and digital pathology: challenges and opportunities
.
J Pathol Inform
.
2018
;
9
:
38
. .
11.
Bera
K
,
Schalper
KA
,
Rimm
DL
,
Velcheti
V
,
Madabhushi
A
.
Artificial intelligence in digital pathology: new tools for diagnosis and precision oncology
.
Nat Rev Clin Oncol
.
2019 Nov
;
16
(
11
):
703
15
. .
12.
Chang
HY
,
Jung
CK
,
Woo
JI
,
Lee
S
,
Cho
J
,
Kim
SW
,
Artificial intelligence in pathology
.
J Pathol Transl Med
.
2019 Jan
;
53
(
1
):
1
12
. .
13.
Colling
R
,
Pitman
H
,
Oien
K
,
Rajpoot
N
,
Macklin
P
; CM‐Path AI in Histopathology Working Group, .
Artificial intelligence in digital pathology: a roadmap to routine use in clinical practice
.
J Pathol
.
2019 Oct
;
249
(
2
):
143
50
. .
14.
Serag
A
,
Ion-Margineanu
A
,
Qureshi
H
,
McMillan
R
,
Saint Martin
MJ
,
Diamond
J
,
Translational AI and deep learning in diagnostic pathology
.
Front Med
.
2019
;
6
:
185
. .
15.
Rossi
ED
,
Schmitt
F
.
Pre-analytic steps for molecular testing on thyroid fine-needle aspirations: the goal of good results
.
Cytojournal
.
2013 Nov 28
;
10
:
24
. .
16.
Layfield
LJ
,
Roy-Chowdhuri
S
,
Baloch
Z
,
Ehya
H
,
Geisinger
K
,
Hsiao
SJ
,
Utilization of ancillary studies in the cytologic diagnosis of respiratory lesions: the papanicolaou society of cytopathology consensus recommendations for respiratory cytology
.
Diagn Cytopathol
.
2016 Dec
;
44
(
12
):
1000
9
. .
17.
Jain
D
,
Roy-Chowdhuri
S
.
Molecular pathology of lung cancer cytology specimens: a concise review
.
Arch Pathol Lab Med
.
2018 Sep
;
142
(
9
):
1127
33
. .
18.
Lozano
MD
,
Echeveste
JI
,
Abengozar
M
,
Mejías
LD
,
Idoate
MA
,
Calvo
A
,
Cytology smears in the era of molecular biomarkers in non-small cell lung cancer: doing more with less
.
Arch Pathol Lab Med
.
2018 Mar
;
142
(
3
):
291
8
. .
19.
Roy-Chowdhuri
S
,
Dacic
S
,
Ghofrani
M
,
Illei
PB
,
Layfield
LJ
,
Lee
C
,
Collection and handling of thoracic small biopsy and cytology specimens for ancillary studies: guideline from the college of American Pathologists in Collaboration with the American College of chest physicians, Association for Molecular Pathology, American Society of Cytopathology, American Thoracic Society, Pulmonary Pathology Society, Papanicolaou Society of Cytopathology, Society of Interventional Radiology, and Society of thoracic radiology
.
Arch Pathol Lab Med
.
2020 May 13
. .
20.
Cosatto
E
,
Laquerre
PF
,
Malon
C
,
Graf
HP
,
Saito
A
,
Kiyuna
T
,
Automated gastric cancer diagnosis on H and E-stained sections; training a classifier on large scale with multiple instance machine learning
.
Medical Imaging 2013: Digital Pathology
.
2013
;
8676
:
867605
.
21.
Malon
CD
,
Cosatto
E
.
Classification of mitotic figures with convolutional neural networks and seeded blob features
.
J Pathol Inform
.
2013
;
4
:
9
. .
22.
Nadeem
A
,
Bhuyan
M
,
Shaik
A
.
A review on semantic segmentation from a modern perspective
. In:
2019 International Conference on Electrical, Electronics and computer engineering (UPCON)
.
Aligarh, India
;
2019
. p.
1
6
.
23.
Ronneberger
O
,
Fischer
P
,
Brox
T
.
U-net: convolutional networks for biomedical image segmentation
.
International Conference on Medical image computing and computer-assisted intervention
.
Cham
:
Springer
;
2015
. p.
234
41
.
24.
Landau
MS
,
Pantanowitz
L
.
Artificial intelligence in cytopathology: a review of the literature and overview of commercial landscape
.
J Am Soc Cytopathol
.
2019 Jul–Aug
;
8
(
4
):
230
41
. .
25.
Suzuki
S
,
Komori
M
,
Hirai
M
,
Ureshino
N
,
Kimura
S
.
Development of a novel, fully-automated genotyping system: principle and applications
.
Sensors
.
2012 Dec 3
;
12
(
12
):
16614
27
. .
26.
Suzuki
S
,
Matsusaka
S
,
Hirai
M
,
Shibata
H
,
Takagi
K
,
Mizunuma
N
,
A novel approach to detect KRAS/BRAF mutation for colon cancer: highly sensitive simultaneous detection of mutations and simple pre-treatment without DNA extraction
.
Int J Oncol
.
2015 Jul
;
47
(
1
):
97
105
. .
27.
Roy Chowdhuri
S
,
Hanson
J
,
Cheng
J
,
Rodriguez-Canales
J
,
Fetsch
P
,
Balis
U
,
Semiautomated laser capture microdissection of lung adenocarcinoma cytology samples
.
Acta Cytol
.
2012
;
56
(
6
):
622
31
. .
Open Access License / Drug Dosage / Disclaimer
This article is licensed under the Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC). Usage and distribution for commercial purposes requires written permission. Drug Dosage: The authors and the publisher have exerted every effort to ensure that drug selection and dosage set forth in this text are in accord with current recommendations and practice at the time of publication. However, in view of ongoing research, changes in government regulations, and the constant flow of information relating to drug therapy and drug reactions, the reader is urged to check the package insert for each drug for any changes in indications and dosage and for added warnings and precautions. This is particularly important when the recommended agent is a new and/or infrequently employed drug. Disclaimer: The statements, opinions and data contained in this publication are solely those of the individual authors and contributors and not of the publishers and the editor(s). The appearance of advertisements or/and product references in the publication is not a warranty, endorsement, or approval of the products or services advertised or of their effectiveness, quality or safety. The publisher and the editor(s) disclaim responsibility for any injury to persons or property resulting from any ideas, methods, instructions or products referred to in the content or advertisements.