Zeitschrift für medizinische Diagnosemethoden
Offener Zugang
ISSN: 2168-9784
ISSN: 2168-9784
Chodakiewitz YG*, Maya MM, Pressman BD
Purpose: To evaluate a recently FDA-approved AI-based radiology workflow triage device for accuracy in identification of intracranial hemorrhage (ICH) on prescreening of real-world CT brain scans.
Method/Materials: An AI-based device ("the algorithm") for ICH detection on CT brain scans was studied at our institution; the algorithm was developed by outside company Aidoc (Tel Aviv, Israel). A retrospective dataset of 533 non-contrast head CT scans was collected from our large urban tertiary academic medical center. Following convention for studies evaluating sensitivities and specificities of imaging computer-aided detection and diagnosis devices, a prevalence-enriched dataset was utilized such that a 50% prevalence of intracranial hemorrhage was obtained. The algorithm was run on the dataset. Cases flagged by the algorithm as positive for ICH were defined as “positive”, and the rest as “negative”. The results were compared to the ground truth, determined by neuroradiologist review of the dataset. Sensitivity and specificity were calculated. Additionally, Negative-Predictive-Value (NPV) and Positive-Predictive-Value (PPV) calculations were made from the prevalence-enriched study data, which enable lower and upper threshold estimates for real-world NPV and PPV, respectively. Metrics were analyzed using a two-sided, exact binomial, 95% confidence-interval.
Results: Algorithm sensitivity was 96.2% (CI: 93.2%-98.2%); specificity was 93.3% (CI: 89.6-96.0%). Estimated realworld NPV was determined as at least 96.2% (CI: 93.2%-97.9%), and an estimated upper threshold for PPV was estimated as 93.4% (CI: 90.1%-95.7%).
Conclusion: The tested device detects intracranial hemorrhage with high sensitivity and specificity. These findings support the potential utility of using the device to autonomously surveil radiology worklists for studies containing critical findings, triage a busy workflow and ultimately improve patient care in clinically time-sensitive cases.