Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules

Pierre P. Massion, Sanja Antic, Sarim Ather, Carlos Arteta, Jan Brabec, Heidi Chen, Jerome Declerck, David Dufek, William Hickes, Timor Kadir, Jonas Kunst, Bennett A. Landman, Reginald F. Munden, Petr Novotny, Heiko Peschl, Lyndsey C. Pickup, Catarina Santos, Gary T. Smith, Ambika Talwar, Fergus Gleeson

Research output: Contribution to journalArticlepeer-review

56 Scopus citations


Rationale: The management of indeterminate pulmonary nodules (IPNs) remains challenging, resulting in invasive procedures and delays in diagnosis and treatment. Strategies to decrease the rate of unnecessary invasive procedures and optimize surveillance regimens are needed. Objectives: To develop and validate a deep learning method to improve the management of IPNs. Methods:ALung Cancer Prediction Convolutional Neural Network model was trained using computed tomography images of IPNs from the National Lung Screening Trial, internally validated, and externally tested on cohorts from two academic institutions. Measurements and Main Results: The areas under the receiver operating characteristic curve in the external validation cohorts were 83.5% (95% confidence interval [CI], 75.4 90.7%) and 91.9% (95% CI, 88.7 94.7%), compared with 78.1% (95% CI, 68.7 86.4%) and 81.9 (95% CI, 76.1 87.1%), respectively, for a commonly used clinical risk model for incidental nodules. Using 5% and 65% malignancy thresholds defining low- and high-risk categories, the overall net reclassifications in the validation cohorts for cancers and benign nodules compared with the Mayo model were 0.34 (Vanderbilt) and 0.30 (Oxford) as a rule-in test, and 0.33 (Vanderbilt) and 0.58 (Oxford) as a rule-out test. Compared with traditional risk prediction models, the Lung Cancer Prediction Convolutional Neural Network was associated with improved accuracy in predicting the likelihood of disease at each threshold of management and in our external validation cohorts. Conclusions: This study demonstrates that this deep learning algorithm can correctly reclassify IPNs into low- or high-risk categories in more than a third of cancers and benign nodules when compared with conventional risk models, potentially reducing the number of unnecessary invasive procedures and delays in diagnosis.

Original languageEnglish (US)
Pages (from-to)241-249
Number of pages9
JournalAmerican journal of respiratory and critical care medicine
Issue number2
StatePublished - Jul 15 2020


  • Computer-aided image analysis
  • Early detection
  • Lung cancer
  • Neural networks
  • Risk stratification

ASJC Scopus subject areas

  • Pulmonary and Respiratory Medicine
  • Critical Care and Intensive Care Medicine


Dive into the research topics of 'Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules'. Together they form a unique fingerprint.

Cite this