Study Highlights Variability in AI Software for Lung Cancer Detection on Chest X-rays

A recent analysis, published in Radiology, highlights the considerable variability in performance among seven AI software platforms designed for the detection of lung cancer using chest X-rays. The retrospective study involved 5,235 patients, with a median age of 60, and indicated lung cancer in 1.4% of the participants. The investigation assessed each platform's sensitivity and specificity for identifying lung cancer.

The study reviewed the performance of AI models, including Annalise Enterprise CXR, ChestView, InferRead DR Chest, TechCare Chest, ChestEye, qXR, and Rayscape CXR. While all platforms have obtained the CE mark, only qXR has received clearance from the Food and Drug Administration (FDA).

The findings revealed sensitivity rates from 20.8% to 77.8% across the AI platforms. Specificity varied from 58.9% to 98.4%, and positive predictive value (PPV) ranged from 1.5% to 28.4%. Lead author Ahmed Maiter, MB BChir, MA, FACR, from the Department of Radiology at Sheffield Teaching Hospitals, and colleagues noted inconsistencies in diagnostic accuracy among devices and poor agreement with traditional radiologist reports.

False positives were a notable issue, ranging from 10 to 2,039 additional detections compared to radiologist assessments. Maiter and colleagues emphasized concerns about automation bias potentially contributing to these elevated false positive rates, as radiologists might hesitate to contradict AI findings.

The study stresses the importance of matching AI platform selection with specific clinical objectives. The choice of software should be guided by its intended clinical use: high-specificity tools may be better for triaging patients directly to CT imaging, while high-sensitivity tools could enhance worklist prioritization or support radiologist accuracy.

The researchers call for further comparative studies to better evaluate different AI platforms for lung cancer detection on chest X-rays and advise health care institutions to carefully consider the deployment purpose of these technologies for optimal integration into existing workflows.