Validation and application of an artificial intelligence robot assisted diagnosis system for diabetic retinopathy

Objective

To evaluate the performance of an artificial intelligence (AI) assisted diagnosis system for diabetic retinopathy (DR) based on deep learning theory.

Methods

Diagnostic performance of a robot assisted diagnosis system called SongYue for DR was trained by using 25 297 retinal images tagged by fundus doctors from multiple hospitals in China.Four types of DR detection model consisting of abnormal DR, referable DR, severe non-proliferative and proliferative DR as well as proliferative DR according to fundus leisions identification were established.The ability of the system to distinguish DR was determined by using receiver operator characteristic (ROC) analysis, sensitivity and specificity of the system.

Results

SongYue system achieved an area under the ROC curve (AUC) of 0.920 for successfully distinguishing normal images from those DR with a sensitivity of 96.0% at a specificity of 87.9%.The AUC of SongYue for referable DR was 0.925, sensitivity was 90.4%, and specificity was 95.2%.For severe non-proliferative and proliferative DR, AUC was 0.845, sensitivity was 72.7%, and specificity was 96.2%.For proliferative DR, AUC was 0.855, sensitivity was 73.5%, and specificity was 97.3%.

Conclusions

SongYue robot assisted diagnosis system has high AUC, sensitivity and specificity for identifying DR, showing good clinical applicable benefits.