PB1019 - Real-time Identification of Patients Diagnosed with Pulmonary Embolism in the Emergency Department using Automated Natural Language Processing of Computed Tomography Reports

Monday, June 26, 2023

18:30 – 19:30 ET

Room: Exhibition

Co-Author(s)

WS

William B. Stubblefield, MD, MPH

Assistant Professor of Emergency Medicine
Vanderbilt University Medical Center
Nashville, Tennessee, United States

Presenting Author(s)

Alyssa N. Altheimer (she/her/hers)

Medical Student
Vanderbilt University School of Medicine
Nashville, Tennessee, United States

Co-Author(s)

SO

Sarah Ouadah

Medical Student
Vanderbilt University School of Medicine
Nashville, Tennessee, United States
ZZ

Zhao Zuo

Analyst
Vanderbilt University Medical Center
Nashville, Tennessee, United States
DG

Dario Giuse

Associate Professor
Vanderbilt University Medical Center
Nashville, Tennessee, United States
JW

Jesse Wrenn

Assistant Professor
Vanderbilt University Medical Center
Nashville, Tennessee, United States
JB

Joshua Beckman

Professor
UTSW Medical Center
Dallas, Texas, United States
SC

Sean Collins

Professor
Vanderbilt University Medical Center
Nashville, Tennessee, United States
Christopher Kabrhel, MD MPH (he/him/his)

Director, Attending Physician, MGH Endowed Chair, Professor of Emergency Medicine
Center for Vascular Emergencies, Department of Emergency Medicine, Massachusetts General Hospital, Harvard Medical School
Boston, Massachusetts, United States

Background: In the United States, the majority of pulmonary emboli (PE) are diagnosed in the emergency department (ED), and most diagnoses are based on computed tomography (CT) results. Prospective screening of patients diagnosed with PE for enrollment into clinical trials and observational studies is traditionally completed by research personnel or clinical staff. This task can be inconsistent, time-intensive, and expensive.

Aims: To derive an automated, natural language processing (NLP) tool and validate its accuracy in providing real-time identification of adult ED patients diagnosed with PE on CT.

Methods: We constructed an expression-based NLP tool through an iterative process using both positive and negative language specific to PE. The tool was applied to CT reports generated by radiologists in the electronic medical record (EMR) in an ED at an academic medical center in Nashville, Tennessee. Once detected by our NLP tool, an alert was sent via an electronic pager system to research personnel notifying them of a PE diagnosis in the ED. This real-time NLP tool was compared to all CT reports (n=2977) manually adjudicated for the presence or absence of PE during a one-year period (10/1/21 – 9/30/22).

Results: There was 95.1% agreement between the NLP tool and CT scans with confirmed PE during the study period (kappa = 0.68). The tool identified 175 PE compared to 317 PE confirmed by manual chart review. This yielded a sensitivity of 54.5%; specificity of 99.9%; positive predictive value (PPV) of 98.9% and negative predictive value (NPV) of 94.8%.

Conclusion(s): This early-stage NLP tool provides proof of concept for automated, real-time identification of PE in the ED. Although there was high agreement between the tool and manual chart adjudication, many PE diagnosed by CT were missed. The miss rate was largely due to NLP application to a specialized segment of CT reports. Efforts to improve sensitivity are underway.