Section: Medical Artificial Intelligence

Purpose : The EviRed project aims to improve the management of diabetic retinopathy (DR), by taking advantage of new imaging modalities and artificial intelligence. In this study, we investigate the fusion of different modalities acquired simultaneously with a PLEX® Elite 9000 (Carl Zeiss Meditec Inc. Dublin, California, USA), namely 3-D structural optical coherence tomography (OCT), 3-D OCT angiography (OCTA) and 2-D Line Scanning Ophthalmoscope (LSO), for the automatic detection of proliferative DR.

Methods : 151 OCT volumes from 64 diabetic patients were collected for the study. This collection was divided as follows: 88 acquisitions (from 31 patients) for training, 28 acquisitions (from 14 patients) for validation and 35 acquisitions (from 19 patients) for testing. DR severity level, according to the ICDR scale, was graded by a retina specialist using fundus photographs: 30 acquisitions (including 16 in the train set, 5 in the validation set and 9 in the test set) had proliferative DR. Three fusion methods were evaluated: Early Fusion, Intermediate Fusion and Hierarchical Fusion. For each method, the following backbones were investigated: Resnet50, Resnet101, Densenet121, and Densenet169. We used the Area under the ROC Curve (AUC) to assess classification performance. These fusion methods were compared to the classification of a single modality separately.

Results : Using a single modality, the structure data achieved the best performance: AUC reaches 0.859 using Resnet101; this is our baseline. The Flow data reached an AUC of 0.816, using Densenet169. The LSO data reached an AUC of 0.662, using Densenet121. Hierarchical fusion achieves the best results, AUC reaches 0.911 using Densenet121, with an increase in AUC of over 0.052 compared to baseline. While the AUC increase for Early Fusion, on Densenet121, was around 0.006. Intermediate Fusion on Densenet121 performed worse than baseline.

Conclusions : The results show the effectiveness of our multimodal fusion method for proliferative DR detection. Hierarchical fusion method is currently showing good results: in particular, it outperforms the detection in a single modality. However, these experiments will have to be replicated in a larger dataset to achieve clinically useful detection performance.