TY - JOUR
T1 - Artificial Intelligence Tool for Assessment of Indeterminate Pulmonary Nodules Detected with CT
AU - Kim, Roger Y.
AU - Oke, Jason L.
AU - Pickup, Lyndsey C.
AU - Munden, Reginald F.
AU - Dotson, Travis L.
AU - Bellinger, Christina R.
AU - Cohen, Avi
AU - Simoff, Michael J.
AU - Massion, Pierre P.
AU - Filippini, Claire
AU - Gleeson, Fergus V.
AU - Vachani, Anil
N1 - Publisher Copyright:
© RSNA, 2022.
PY - 2022/9
Y1 - 2022/9
N2 - Background Limited data are available regarding whether computer-aided diagnosis (CAD) improves assessment of malignancy risk in indeterminate pulmonary nodules (IPNs). Purpose To evaluate the effect of an artificial intelligence-based CAD tool on clinician IPN diagnostic performance and agreement for both malignancy risk categories and management recommendations. Materials and Methods This was a retrospective multireader multicase study performed in June and July 2020 on chest CT studies of IPNs. Readers used only CT imaging data and provided an estimate of malignancy risk and a management recommendation for each case without and with CAD. The effect of CAD on average reader diagnostic performance was assessed using the Obuchowski-Rockette and Dorfman-Berbaum-Metz method to calculate estimates of area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Multirater Fleiss κ statistics were used to measure interobserver agreement for malignancy risk and management recommendations. Results A total of 300 chest CT scans of IPNs with maximal diameters of 5-30 mm (50.0% malignant) were reviewed by 12 readers (six radiologists, six pulmonologists) (patient median age, 65 years; IQR, 59-71 years; 164 [55%] men). Readers' average AUC improved from 0.82 to 0.89 with CAD (
P < .001). At malignancy risk thresholds of 5% and 65%, use of CAD improved average sensitivity from 94.1% to 97.9% (
P = .01) and from 52.6% to 63.1% (
P < .001), respectively. Average reader specificity improved from 37.4% to 42.3% (
P = .03) and from 87.3% to 89.9% (
P = .05), respectively. Reader interobserver agreement improved with CAD for both the less than 5% (Fleiss κ, 0.50 vs 0.71;
P < .001) and more than 65% (Fleiss κ, 0.54 vs 0.71;
P < .001) malignancy risk categories. Overall reader interobserver agreement for management recommendation categories (no action, CT surveillance, diagnostic procedure) also improved with CAD (Fleiss κ, 0.44 vs 0.52;
P = .001). Conclusion Use of computer-aided diagnosis improved estimation of indeterminate pulmonary nodule malignancy risk on chest CT scans and improved interobserver agreement for both risk stratification and management recommendations. © RSNA, 2022
Online supplemental material is available for this article. See also the editorial by Yanagawa in this issue.
AB - Background Limited data are available regarding whether computer-aided diagnosis (CAD) improves assessment of malignancy risk in indeterminate pulmonary nodules (IPNs). Purpose To evaluate the effect of an artificial intelligence-based CAD tool on clinician IPN diagnostic performance and agreement for both malignancy risk categories and management recommendations. Materials and Methods This was a retrospective multireader multicase study performed in June and July 2020 on chest CT studies of IPNs. Readers used only CT imaging data and provided an estimate of malignancy risk and a management recommendation for each case without and with CAD. The effect of CAD on average reader diagnostic performance was assessed using the Obuchowski-Rockette and Dorfman-Berbaum-Metz method to calculate estimates of area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. Multirater Fleiss κ statistics were used to measure interobserver agreement for malignancy risk and management recommendations. Results A total of 300 chest CT scans of IPNs with maximal diameters of 5-30 mm (50.0% malignant) were reviewed by 12 readers (six radiologists, six pulmonologists) (patient median age, 65 years; IQR, 59-71 years; 164 [55%] men). Readers' average AUC improved from 0.82 to 0.89 with CAD (
P < .001). At malignancy risk thresholds of 5% and 65%, use of CAD improved average sensitivity from 94.1% to 97.9% (
P = .01) and from 52.6% to 63.1% (
P < .001), respectively. Average reader specificity improved from 37.4% to 42.3% (
P = .03) and from 87.3% to 89.9% (
P = .05), respectively. Reader interobserver agreement improved with CAD for both the less than 5% (Fleiss κ, 0.50 vs 0.71;
P < .001) and more than 65% (Fleiss κ, 0.54 vs 0.71;
P < .001) malignancy risk categories. Overall reader interobserver agreement for management recommendation categories (no action, CT surveillance, diagnostic procedure) also improved with CAD (Fleiss κ, 0.44 vs 0.52;
P = .001). Conclusion Use of computer-aided diagnosis improved estimation of indeterminate pulmonary nodule malignancy risk on chest CT scans and improved interobserver agreement for both risk stratification and management recommendations. © RSNA, 2022
Online supplemental material is available for this article. See also the editorial by Yanagawa in this issue.
KW - Aged
KW - Artificial Intelligence
KW - Female
KW - Humans
KW - Lung Neoplasms/diagnostic imaging
KW - Male
KW - Multiple Pulmonary Nodules/diagnostic imaging
KW - Retrospective Studies
KW - Sensitivity and Specificity
KW - Tomography, X-Ray Computed/methods
UR - http://www.scopus.com/inward/record.url?scp=85137008002&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85137008002&partnerID=8YFLogxK
U2 - 10.1148/radiol.212182
DO - 10.1148/radiol.212182
M3 - Article
C2 - 35608444
AN - SCOPUS:85137008002
SN - 0033-8419
VL - 304
SP - 683
EP - 691
JO - Radiology
JF - Radiology
IS - 3
ER -