An AI tool used to detect pulmonary embolism on CT pulmonary angiograms showed high agreement with radiologists in a large real-world study published in Radiology: Artificial Intelligence.
Researchers at Northwell Health reviewed 32,501 CT pulmonary angiograms performed across the health system over 18 months. The AI software, developed by Aidoc, matched radiologist interpretations in 97.8% of cases, according to the study.
Agreement was higher for negative scans than positive scans, with concordance rates of 98.18% and 93.75%, respectively. The findings suggest the algorithm may be particularly useful for helping rule out pulmonary embolism while supporting radiology triage workflows.
The study also highlighted limitations of relying solely on AI interpretation. Among confirmed pulmonary embolism cases, 15% were identified by radiologists but missed by the algorithm.
Researchers independently reviewed all cases in which the AI output and radiologist interpretation differed. Thoracic radiologists determined that radiologists were correct in 88.7% of discordant cases, while AI was correct in 11.3%.
In cases where the AI flagged pulmonary embolism but the radiologist did not, adjudicators agreed with the radiologist 85.6% of the time. In cases where the AI missed pulmonary embolism identified by radiologists, adjudicators sided with radiologists in 89.8% of cases.
“AI-informed radiologists achieved a sensitivity of 99.2% for pulmonary embolism detection,” said Dr. Shlomit Goldberg-Stein, professor of radiology and director of artificial intelligence at the Zucker School of Medicine at Hofstra/Northwell, in a statement.
Pulmonary embolism remains a major cause of mortality, accounting for an estimated 5% to 10% of in-hospital deaths in the United States.
The authors said the findings support continued use of human oversight alongside AI systems in clinical imaging workflows. Dr. Matthew Barish, vice chair of radiology informatics at the Zucker School of Medicine at Hofstra/Northwell, said the results demonstrate “the value of AI-triage while also demonstrating the continued role of the radiologist in the clinical pathway.”