Multi-Modal Learning Using Physicians Diagnostics for Optical Coherence Tomography Classification

Yash Yee Logan, Kiran Kokilepersaud, Gukyeong Kwon, Ghassan Alregib, Charles Wykoff, Hannah Yu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we propose a framework that incorporates experts diagnostics and insights into the analysis of Optical Coherence Tomography (OCT) using multi-modal learning. To demonstrate the effectiveness of this approach, we create a medical diagnostic attribute dataset to improve disease classification using OCT. Although there have been successful attempts to deploy machine learning for disease classification in OCT, such methodologies lack the experts insights. We argue that injecting ophthalmological assessments as another supervision in a learning framework is of great importance for the machine learning process to perform accurate and interpretable classification. We demonstrate the proposed framework through comprehensive experiments that compare the effectiveness of combining diagnostic attribute features with latent visual representations and show that they surpass the state-of-the-art approach. Finally, we analyze the proposed dual-stream architecture and provide an insight that determine the components that contribute most to classification performance.

Original languageEnglish (US)
Title of host publicationISBI 2022 - Proceedings
Subtitle of host publication2022 IEEE International Symposium on Biomedical Imaging
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781665429238
DOIs
StatePublished - 2022
Event19th IEEE International Symposium on Biomedical Imaging, ISBI 2022 - Kolkata, India
Duration: Mar 28 2022Mar 31 2022

Publication series

NameProceedings - International Symposium on Biomedical Imaging
Volume2022-March
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452

Conference

Conference19th IEEE International Symposium on Biomedical Imaging, ISBI 2022
Country/TerritoryIndia
CityKolkata
Period3/28/223/31/22

Keywords

  • Autoencoder
  • Diagnostic Attributes
  • Latent Representation
  • Multi-modal Learning
  • OCT

ASJC Scopus subject areas

  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging

Fingerprint

Dive into the research topics of 'Multi-Modal Learning Using Physicians Diagnostics for Optical Coherence Tomography Classification'. Together they form a unique fingerprint.

Cite this