Interpretable multimodal classification for age-related macular degeneration diagnosis

PLoS One. 2024 Nov 11;19(11):e0311811. doi: 10.1371/journal.pone.0311811. eCollection 2024.

Abstract

Explainable Artificial Intelligence (XAI) is an emerging machine learning field that has been successful in medical image analysis. Interpretable approaches are able to "unbox" the black-box decisions made by AI systems, aiding medical doctors to justify their diagnostics better. In this paper, we analyze the performance of three different XAI strategies for medical image analysis in ophthalmology. We consider a multimodal deep learning model that combines optical coherence tomography (OCT) and infrared reflectance (IR) imaging for the diagnosis of age-related macular degeneration (AMD). The classification model is able to achieve an accuracy of 0.94, performing better than other unimodal alternatives. We analyze the XAI methods in terms of their ability to identify retinal damage and ease of interpretation, concluding that grad-CAM and guided grad-CAM can be combined to have both a coarse visual justification and a fine-grained analysis of the retinal layers. We provide important insights and recommendations for practitioners on how to design automated and explainable screening tests based on the combination of two image sources.

MeSH terms

  • Artificial Intelligence
  • Deep Learning
  • Humans
  • Image Processing, Computer-Assisted / methods
  • Macular Degeneration* / classification
  • Macular Degeneration* / diagnosis
  • Macular Degeneration* / diagnostic imaging
  • Multimodal Imaging / methods
  • Retina / diagnostic imaging
  • Retina / pathology
  • Tomography, Optical Coherence* / methods

Grants and funding

The authors gratefully acknowledge financial support from ANID PIA/PUENTE AFB230002 and FONDECYT-Chile, grants 12200007, 1200221, 11191215, and 1212038. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.