Computer-Aided Diagnosis Based on Convolutional Neural Network System for Colorectal Polyp Classification: Preliminary Experience

It is now generally accepted that most colorectal cancers originate from adenomas [1,2,3]. This notion, called the “adenoma-carcinoma sequence,” has been supported by clinical observational studies in which early detection and removal of adenomas efficiently prevented the development of colon cancers [1,2,3]. The majority of small colorectal polyps are hyperplastic polyps with little or no ability to differentiate into colorectal cancer [4]. Therefore, accurate and objective diagnosis of small colorectal polyps reduces unnecessary biopsies and endoscopic resections, which helps avoid the complications associated with endoscopic procedures. However, it is sometimes difficult even for well-experienced endoscopists to discriminate between hyperplastic polyps and adenomas by conventional white-light observation, image-enhanced endoscopy, and chromoendoscopy [5]. Establishment of an accurate and objective diagnostic tool for the classification of colon polyps would be very useful for both experienced endoscopists and apprentices.

Computer-aided diagnosis (CAD) is becoming a next-generation tool for the diagnosis of human disease [6]. The CAD of colon polyps has been reported on previously [7,8,9,10,11,12,13,14,15,16]. Previous studies have reported the usefulness of CAD for colon polyps. CAD for colon polyps has been suggested as a particularly useful tool for trainee colonoscopists, as the use of a CAD system helps avoid the complications associated with endoscopic resections [9,10,11]. Moreover, the higher rate of correct diagnoses using a CAD system also suggests the potential for this diagnostic tool to improve the quality of colonoscopic examinations.

In addition to conventional CAD, a convolutional neural network (CNN) system utilizing artificial intelligence (AI) has been developing rapidly over the past 5 years. This system is widely used to recognize human faces on Facebook and in the Google images tool [17]. We hypothesized that a CNN system based on CAD utilizing AI might achieve a higher diagnostic accuracy than conventional CAD without CNN or AI for colorectal polyps. Hence, we attempted to generate a unique CNN-CAD system with an AI function that studied endoscopic images extracted from movies obtained with colonoscopes used in routine examinations. Here, we report our preliminary results of this novel CNN-CAD system for the diagnosis of colon polyps.

A total of 1,200 images from cases of colonoscopy performed between January 2010 and December 2016 at Kindai University Hospital were used. These images were extracted from the video of actual endoscopic examinations. Additional video images from 10 cases of unlearned processes were retrospectively assessed in a pilot study. They were simply diagnosed as either an adenomatous or nonadenomatous polyp. This was a collaborative study between the Department of Gastroenterology and Hepatology, Faculty of Medicine, and the Faculty of Science and Engineering at Kindai University. This study was approved by the Ethics Committee of the Kindai University Faculty of Medicine. The procedures followed were in accordance with the guidelines of the World Medical Association's Declaration of Helsinki.

CNN is a type of artificial neural network that imitates the function of the receptive fields in the human brain. A CNN is one of the major algorithms in deep learning and is a state-of-the-art method in image and speech recognition.

The functional architecture of a CNN is illustrated in Figure 1: the CNN perceives an image as an input, and the output is a class label corresponding to the input image. For instance, the class labels in this study consist of “adenomatous” and “nonadenomatous.” As depicted in Figure 1, there are several kinds of layers in the CNN. Each neuron in a layer is connected to the corresponding neurons in the previous layer.

An input image is presented to the first layer, i.e., the left side of Figure 1. The number of neurons in the first layer is equal to the number of pixels in the input image (in a gray-scaled image) or 3 times the number of pixels (in a color image). There are 2 kinds of information processing which are applied iteratively: convolution and pooling.

The convolution process works as a filter that extracts features from images or data in the previous layer. Note that Figure 1 is just a simplified explanation of a CNN. There are a large number of filters in the convolution that are applied simultaneously. The parameter of the filters, which defines the feature to be extracted, is adjusted by learning algorithms. The size of the filters is smaller than that of the layer, so that the filter is repeatedly applied within a stratum.

The pooling process selects the strongest activated value for a feature in a local area that is extracted in the convolution. Through pooling, even if the image is shifted slightly, the classification results are not affected and the size of the layer is reduced by a quarter, as delineated in Figure 1.

For the final layer, all the neurons in one layer are connected to all the neurons in the previous layer. For each connection, a parameter is associated that is also adjusted by learning. The number of neurons in the final layer is equivalent to the number of class labels to be recognized. By using the softmax function, the output of the CNN can be represented as a probability.

The number of images used by AI to learn to distinguish adenomatous from nonadenomatous was 1,200:600. These images were extracted from the videos of actual endoscopic examinations. The size of each image was adjusted to 256 × 256 pixels. A 10-hold cross-validation was carried out. The accuracy of the 10-hold cross-validation is 0.751, where the accuracy is the ratio of the number of correct answers over the number of all the answers produced by the CNN. The images used for AI learning include images using conventional white-light and narrow-band imaging (NBI) and chromoendoscopy.

Ten additional images of unlearned endoscopic movies were applied as a test, as shown in Figure 2. These images were not used in the training of the CNN.

The output of the CNN was presented as probabilities: P_A = adenomatous and P_N = nonadenomatous. The output with the higher probability was regarded as a decision by the CNN. The final evaluation of our CNN was to compare the results of pathological examinations. The decisions by the CNN were correct in 7 of 10 cases.

Removal of colon adenomas has been established as the most efficient strategy for the prevention of colorectal cancers [1,2,3]. It should be noted, however, that small colorectal polyps include both adenomatous polyps and nonadenomatous polyps, the latter of which have little or no possibility to differentiate into colorectal cancers [4]. Thus, discrimination of adenomatous polyps from nonadenomatous polyps by endoscopic resection is very important to avoid unnecessary removal of colorectal tumors. Although an accurate diagnosis of adenomatous and nonadenomatous polyps by endoscopic findings is absolutely necessary to avoid unnecessary resection of colon polyps, an accurate endoscopic diagnosis of small colorectal polyps is sometimes difficult even for well-trained colonoscopists. Thus, an accurate and reliable diagnostic system is required to discriminate between adenomatous and nonadenomatous polyps in endoscopic findings.

The use of CAD technology has the potential to become a novel diagnostic tool for colorectal tumors. Automatic diagnosis by computer in the CAD system may increase the diagnostic accuracy and reduce the number of unnecessary biopsies and endoscopic resections. As a result, this diagnostic system has the potential to avoid the complications related to unnecessary endoscopic procedures. Initial studies report the usefulness of CAD for colorectal polyps using still images obtained from magnifying NBI observations [12,13]. As opposed to CAD using still images, 2 Japanese groups have developed CAD with real-time endoscopy [14,15]. These real-time CAD systems are operated during the actual endoscopic observation. Moreover, Japanese researchers have successfully developed an automated CAD using endocytoscopy based on automated extraction of ultra-magnified nuclear features followed by machine-learning analysis from the same group [9,10,11]. Thus, a wide variety of CAD systems are available at present. Importantly, the diagnostic accuracy of these novel methods is higher than that achieved by endoscopic diagnosis by less-experienced endoscopists [9,10,11].

In this study, we tried to establish a novel CAD system based on CNN and AI. A CNN system is one of the major algorithms in deep learning for AI [17]. Deep learning is a more sophisticated machine-learning method used for computer vision. CNN has the advantage of being able to learn from large data and has led to high accuracy and rapid processing time. Although a CNN-CAD system has been employed for colorectal polyps by using still images of chromoendoscopy in 1 report [17], no studies have used images extracted from movies of real-time routine colonoscopy. Thus, to our knowledge, this is the first report on the utility of CNN based on CAD using real-time endoscopy images. This pilot study demonstrates that our newly developed CNN-CAD system with endoscopic movies might be useful as a fully computer-automated instant diagnostic tool for distinguishing neoplastic changes in colorectal lesions. The diagnostic accuracy for distinguishing nonadenomatous from adenomatous polyps was 70%. The diagnostic accuracy for our CNN-CAD system is not satisfactory at present as compared to previous reports [7,8,9,10,11,12,13,14,15,16]. In this regard, we assume that deep learning with 600 or 1,200 images might not be sufficient and, therefore, attribute the relatively low performance to the low number of images.

Alternatively, learning with NBI or pit pattern photographs, rather than conventional white-light photographs, might lead to higher rates of accurate diagnoses, since the former photographs provide clearer features than the latter photographs. In any case, a large number of test cases are absolutely necessary to evaluate the usefulness of our CNN-CAD system. Nonetheless, our CNN-CAD system would still be an attractive choice, since the decision-making process with regard to the removal of colorectal polyps might be simplified by application of this system, especially for less-experienced colonoscopists. Although our CNN-CAD system with colonoscopy videos was applied to discriminate between adenomatous and nonadenomatous polyps in this initial trial, it might be possible to perform differential diagnoses for adenoma, early cancer, and advanced cancer in the future. If our CNN-CAD system can perform an objective evaluation of colorectal tumors, it seems likely that medical malpractice resulting from misdiagnosis could be reduced.

A CNN-CAD system using routine colonoscopy might be useful for the rapid diagnosis of colorectal polyp classification. Further prospective studies in an in vivo setting are required to confirm the effectiveness of a CNN-CAD system in routine colonoscopy.

The authors have no conflicts of interest to declare.