A Preliminary Study of Deep-Learning Algorithm for Analyzing Multiplex Immunofluorescence Biomarkers in Body Fluid Cytology Specimens

Distinguishing metastatic adenocarcinoma from reactive mesothelial cells in body fluid cytology can be challenging. Diagnoses often require the assistance of immunohistochemical (IHC) staining for a battery of biomarkers, including biomarkers for mesothelial cells and markers for metastatic malignant glandular epithelial cells. Since IHC-based staining is usually performed 1 biomarker at a time and involves multiple cell block sections, it is often difficult to align the staining pattern of biomarkers on cells of interest. This is especially problematic in cases with limited cellularity for evaluation.

One potential solution is multiplex fluorescence imaging (MFI) analysis, which allows simultaneous detection of up to 7 targets with quantification capabilities at high resolution on a single cell basis [1, 2]. This approach has been shown to be applicable in a variety of tissue-based settings. However, visual scoring of multiple biomarkers, particularly in a cell block section, can be challenging and time-consuming.

The development of an image-based automatic analytical system that can determine the expression of multiple biomarkers with greater objectivity and reproducibility, and alleviate the burden on pathologists to make challenging and time-consuming assessments, may have significant clinical implications. Deep-learning combined with digital technologies have potential for performing objective, quantitative assessments for biomarker analysis in the single-cell level [3, 4]. These deep-learning approaches use artificial convolutional neural networks, which can be trained directly from primary data without significant pathologist guidance on annotating morphologic or cytologic features. Studies comparing deep-learning based digital image analysis against manual biomarker assessment have been performed in subtyping breast cancer [5], measuring PD-L1 expression in non-small cell lung cancer [6], and scoring tumor infiltrating lymphocytes in tumor areas [7], with deep-learning methods showing superiority or at least non-inferiority to manual scoring with respect to accuracy and reproducibility.

Here, we present the development of a digital image-based deep-learning algorithm (DLA) for analyzing biomarkers in MFI settings (MFI-DLA). In particular, we report the feasibility of performing DLA on cell block sections of pleural fluid samples.

We retrospectively and randomly selected 41 out of a total of 864 pleural fluid cytology specimens from archived cell block samples between 2015 and 2018. Of the 41 pleural fluid specimens selected, 20 were malignant and 21 were benign. All samples were collected in the cytology laboratory at University of California, Los Angeles (UCLA). Patient primary histological and cytology diagnosis were recorded for the analysis. Cases with an equivocal cytology diagnosis (atypical or suspicious) were not included in the study. The study was exempt from UCLA’s Institutional Review Board assessment as archived cell blocks were used and no patient identifiers were recorded.

Pleural effusion samples were each centrifuged at 2,500 rpm for 10 min. Supernatant fluid was discarded and the cell pellet transferred to a 1.5-mL centrifugation tube. One drop of warmed agar was added to each pellet and the samples were quickly agitated. Clots formed within 30–60 s and each sample was subsequently placed in a tissue cassette and processed as routine histology samples.

The paraffin slides were stained using the Opal^TM Multiplex IHC Kit (Akoya Bioscences), which utilizes a sequence of antibody-antigen binding, fluorescence staining using HRP-conjugated secondary antibody, and heat-mediated removal of the target-binding complex for subsequent targeting of a different antigenic biomarker. In addition to the greater degree of multiplexing, this kit allows for compatibility with unlabeled primary antibodies, reduced cross-reactivity between antibodies, and no limitations with species compatibility by virtue of serial biomarker staining.

Briefly, cell block sections were dewaxed with xylene, rehydrated with a series of ethanol solutions, and washed. Antigen retrieval was performed using microwave treatment (45 s at 100% power, then additional 15 min at 20% power) with AR buffer (AR6 or AR9, Akoya Bioscences), then the slides were blocked, cooled, and washed with distilled water and TBST. Slides were incubated with primary antibodies specific to calretinin (Invitrogen, Cat#:180211, 1:100 dilution), Moc31 (Dako, #M3525, 1:200 dilution), and BerEP4 (Dako, #M0804, 1:200 dilution). Slides were then incubated with secondary anti-rabbit HRP-conjugated antibody (Dako envision, #K4003) or anti-mouse HRP-conjugated antibody (Dako envision, #K4001). Then a signal amplification of Opal fluorophore (1:100 dilution at room temperature for 10 min), followed by additional microwave treatment (45 s at 100% power, then additional 15 min at 20% power) between each antibody staining, and a final application of DAPI counterstain (1:2,000 dilution for 5 min) was performed. The fluorophores used were Opal 520 FITC for calretinin; Opal 570 TRITC for Moc31; and Opal 650 CY5 for BerE4. Fluorescence image exposure times are as follows: DAPI: 3.977 ms, FITC (OPAL 520): calretinin 37.27 ms, Cy3 (OPAL 570): MOC31 35.856 ms, and Cy5 (OPAL 690): BerEP4 39.09 ms. The images were then scanned at ×20 magnification (Vectra Polaris, Akoya) and uploaded to a secure UCLA file share system.

We designed the multiplexed biomarker analysis protocol package (APP) with Visiopharm, a company that specializes in developing image analysis software using DLA (https://www.visiopharm.com). The APP was designed to allow biomarker combinations to be analyzed, provided that the panels (1) used DAPI nuclei stain to highlight nucleated cells, (2) included a total of 4 stains, (3) used identical fluorophores (FITC, TRITC, CY5, and DAPI), and (4) scanned each fluorophore channel in the order performed for images on the training dataset. In addition, while the APP was initially validated on whole ovarian tumor tissue slides cut from formalin-fixed paraffin embedded tissue blocks, the APP has the capability of evaluating multiplexed biomarker expression on cytology specimen preparations such as aspirate smears, cytospins, and cell blocks, so long as the above criteria are met. The APP was also partitioned into several “sub-protocols,” namely tumor/stroma segmentation (not used in the cell block setting), cell identification, and DAPI quantification on a region of interest (for detecting aneuploidy or polyploidy). This separation was implemented to allow the end-user to select functions based on the clinical situation and avoid wasting time on computational tasks that were not needed.

A significant challenge in developing DLA is curating a high-quality training dataset. Traditionally, this training dataset requires a pathologist to manually annotate enough features (roughly 1,000 objects), which is very time-consuming. To solve this challenge, our APP used automatic annotation by a process known as virtual double-staining, which aligns serial H&E and MFI sections so that information on the MFI can be transferred to H&E without manually annotating the H&E. These automatic annotations are then fed into a deep neural network; once trained, the APP can receive a new sample, create a probability map of likely areas of biomarker expression or features of interest, and assign a probability threshold percentage to then perform segmentation or identification (Fig. 1). The end result is a faster, more precise, and scalable curation of training data.

Manual MFI and MFI-DLA used the same criteria to determine if the specimen was “positive” or “negative” for metastatic malignancy: if there were cells which stained positive for MOC31 and/or BerEP4 and negative for calretinin, then the specimen was scored positive. Otherwise, the specimen was considered negative. Cells which stained positive for calretinin (green fluorescence) with additional MOC31 (yellow fluorescence) and/or BerEP4 (red fluorescence) positivity were considered equivocal. MFI analysis was carried out 6 months after manual MFI analysis as a washout period. Diagnoses from visual scoring and DLA scoring were compared with patient’s primary diagnosis and cytology diagnosis. Final results were confirmed by 2 pathologists (J.A.J. and J.Y.R.).

A total of 41 cases of pleural fluids, of which 20 were malignant and 21 were benign, were selected for the study. Seven of the 21 benign cases were noted to have only red blood cells on the cell blocks. These cases were excluded, leaving a total of 14 benign cases. The analysis process and staining patterns of MOC31, BerEP4 and calretinin are shown in Figure 2. DLA scoring generated the same results as the visual scoring (100% concordance, Table 1). While all cases with positive original cytological diagnoses showed positive visual MFI and MFI-DLA scores, 2 of the 14 (14.3%) original cytologically negative cases had rare calretinin positive cells that also stained positive for MOC31 and/or BerEP4. These 2 cases were considered “equivocal” by MFI and MFI-DLA analysis.

While initially validated on histological sections, the APP was designed with enough flexibility for analyzing cytology specimens with different cell markers expressed on heterogeneous cell populations. Here, we reported the results of DLA scoring for analyzing biomarkers in a multiplex immunofluorescence setting. Although only 34 cases were included in final analysis, our study demonstrated the feasibility of MFI-DLA analysis with complete (100%) concordance with manual MFI analysis.

It is interesting that 2 of the original cytologically negative cases had rare cells with MOC31 and/or BerEP4 positivity. This is not surprising as neither MOC31 nor BerEP4 antibody are 100% specific for identifying metastatic adenocarcinoma. MOC31 and BerEP4 expression may be seen in cases of reactive mesothelial cells at frequencies up to 13 and 10%, respectively [8, 9]. Albeit there were occasional nonspecific expressions of MOC31 or BerEP4 in mesothelial cells, a significant finding of 100% concordance between the manual score and DLA score was noted. These findings highlight the potential application of MFI-DLA in determining metastatic malignancies in body fluid cytological specimens.

Another interesting finding was the variable cellularity across the entire dataset; Figure 2 shows a typical positive case. In this study, the percentage of positive tumor cells in relation to mesothelial cells was not measured as it did not affect/contribute the outcome (positive or negative for tumor cells), though we acknowledge that this would be interesting to characterize in a future study.

While the preliminary study demonstrated the potential application of our MFI-DLA in differentiating metastatic malignancy versus benign pleural fluid samples, this approach may be useful in other settings important for diagnosis and management of cancer, such as quantifying PD-L1, Her-2, P16, and Ki-67 expression in single-cell levels. In fact, the program was developed initially for the analysis of PD-L1 in MFI setting. However, carefully designed studies will be needed to validate the value of MFI-DLA in actual clinical settings.

In conclusion, it is feasible to perform MFI-DLA analysis on cell block sections from body fluid samples. While most DLA methods have been validated on histology specimens, there may also be value in applying these tools to cytology specimens in clinical settings where fluids and cellular aspirates are more accessible and convenient to obtain. However, the actual clinical utility of this approach remains to be determined in additional studies with larger sample sizes.

We thank Sarah Holm Jørgensen from Visiopharm for technical expertise and support in APP development.

The project received exemption from Institutional Review Board for human research subjects at UCLA.

The authors have no conflicts of interest to declare.

Project funded internally by the research fund of Department of Pathology and Laboratory Medicine, UCLA.

Jianyu Rao conceptualized the study. Curtis Chin worked with Visiopharm to develop MFI-DLA program, Josephine S. Aguilar-Jakthong and Yunfeng Li prepared the cases and performed the stains and manual analysis. Weibo Yu performed MFI-DLA analysis. Elizabeth Rao, Christine Chow, and Grace Wang were student volunteers in Jianyu Rao’s group who worked with Weibo Yu virtually to carry out MFI-DLA analysis and manuscript preparation.