TY - JOUR
T1 - Accurate prediction of disease-risk factors from volumetric medical scans by a deep vision model pre-trained with 2D scans
AU - Avram, Oren
AU - Durmus, Berkin
AU - Rakocz, Nadav
AU - Corradetti, Giulia
AU - An, Ulzee
AU - Nittala, Muneeswar G.
AU - Terway, Prerit
AU - Rudas, Akos
AU - Chen, Zeyuan Johnson
AU - Wakatsuki, Yu
AU - Hirabayashi, Kazutaka
AU - Velaga, Swetha
AU - Tiosano, Liran
AU - Corvi, Federico
AU - Verma, Aditya
AU - Karamat, Ayesha
AU - Lindenberg, Sophiana
AU - Oncel, Deniz
AU - Almidani, Louay
AU - Hull, Victoria
AU - Fasih-Ahmad, Sohaib
AU - Esmaeilkhanian, Houri
AU - Cannesson, Maxime
AU - Wykoff, Charles C.
AU - Rahmani, Elior
AU - Arnold, Corey W.
AU - Zhou, Bolei
AU - Zaitlen, Noah
AU - Gronau, Ilan
AU - Sankararaman, Sriram
AU - Chiang, Jeffrey N.
AU - Sadda, Srinivas R.
AU - Halperin, Eran
N1 - Publisher Copyright:
© The Author(s), under exclusive licence to Springer Nature Limited 2024.
PY - 2024
Y1 - 2024
N2 - The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for ‘slice integration by vision transformer’), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications.
AB - The application of machine learning to tasks involving volumetric biomedical imaging is constrained by the limited availability of annotated datasets of three-dimensional (3D) scans for model training. Here we report a deep-learning model pre-trained on 2D scans (for which annotated data are relatively abundant) that accurately predicts disease-risk factors from 3D medical-scan modalities. The model, which we named SLIViT (for ‘slice integration by vision transformer’), preprocesses a given volumetric scan into 2D images, extracts their feature map and integrates it into a single prediction. We evaluated the model in eight different learning tasks, including classification and regression for six datasets involving four volumetric imaging modalities (computed tomography, magnetic resonance imaging, optical coherence tomography and ultrasound). SLIViT consistently outperformed domain-specific state-of-the-art models and was typically as accurate as clinical specialists who had spent considerable time manually annotating the analysed scans. Automating diagnosis tasks involving volumetric scans may save valuable clinician hours, reduce data acquisition costs and duration, and help expedite medical research and clinical applications.
UR - http://www.scopus.com/inward/record.url?scp=85205342308&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85205342308&partnerID=8YFLogxK
U2 - 10.1038/s41551-024-01257-9
DO - 10.1038/s41551-024-01257-9
M3 - Article
AN - SCOPUS:85205342308
SN - 2157-846X
JO - Nature Biomedical Engineering
JF - Nature Biomedical Engineering
ER -