Improving the Safety of Diagnosis and Therapy in the Inpatient Setting

Principal Investigators: Anuj K. Dalal, M.D., and David Bates, M.D., MSc., Brigham & Women’s Hospital, Boston, MA
AHRQ Grant No.: HS26613
Project Period: 09/01/18-06/30/22

Description: The overall goal of this learning lab was to improve diagnostic safety and link diagnosis to the correct treatment in acute care. To address this overall issue, the lab used rigorous systems engineering and human factors methods to guide its approach.

The specific aims were to conduct:

1. Problem analysis using systems engineering methods to analyze the problem of diagnostic and therapeutic error and identify system and cognitive factors for a set of morbid, costly common conditions and undifferentiated symptoms.
2. Design and development using human factors methods and rapid, iterative prototyping of interventions to engage care teams and patients/caregivers to ensure treatment trajectories match the anticipated course for working diagnoses or symptoms, ensuring alignment with patient and clinician expectations.
3. Implementation and evaluation by conducting pilot tests, training clinical staff, implementing interventions in the acute care setting, assessing the impact on diagnostic errors that lead to patient harm, and performing quantitative and qualitative evaluations.

This PSLL created multiple interventions to improve diagnostic safety among hospitalized patients. They include^1-4:

• A Diagnostic Timeout (DTO) to assist clinicians in reassessing the diagnosis of patients (i.e., inside or outside their hospital rooms) when risk for diagnostic error (DE) or diagnostic uncertainty is high. It comes in the form of a pocket card, app, and Smart Phrase.
• A Diagnostic Safety Educational Curriculum that comprises short, animated video tutorials delivered online to engage clinician learners in using the DTO to address diagnostic uncertainty and mitigate risk of DE. An interactive newsletter was also created.
• A Diagnostic Safety Column (DSC) and DE Predictive Algorithm housed in the electronic health record and modeled as a DE risk score (i.e., a₁X₁ + a₂X₂ + a₃X₃ + a₄X₄ … + a_nX_n). In this risk score, X_n represents baseline risk factors and in-hospital clinical factors, as well as the coefficient (weight) of each independent variable. Configurable parameters (i.e., variables, weights) enable adjusted baseline risk estimates in real time based on new clinician data collected during hospitalization.
• A Patient Diagnostic Questionnaire (PDQ) that aligns with the “Patient Experience” Safer Dx process dimension to assess patients’ understanding of their diagnosis at optimal times so their feedback can be relayed back to the care team.

Leading to the creation of these interventions, researchers first adapted validated tools (Safer Dx, DEER Taxonomy) to pinpoint 44 diagnostic process failures (DPFs) in a cohort of representative cases.¹ Most of the significant DPFs (63.3%) occurred during the “Diagnostic Information and Patient Follow-up” and “Patient and Provider Encounter and Initial Assessment” phases.

Next, researchers developed a structured case review process that was validated against their institution’s mortality review process. Using this process, they found that harmful DEs were common, occurring in about 7% of hospitalized patients, especially among high-risk patients (i.e., those who were transferred unexpectedly to intensive care, died within 90 days of hospitalization, or had complex clinical events).

To test the DSC and DE predictive algorithm interventions, the lab assembled a stratified, random sampling of 675 patients (339 preimplementation, 336 postimplementation) out of a total of 9,147 possible. Each cohort was based on clinical criteria with varying likelihoods of DE. The high-risk cohorts included cases of patients who were transferred to the ICU after 24 hours, died within 90 days of hospitalization but were not transferred to the ICU, or had complex clinical events based on coded electronic triggers.¹

Researchers found a nonsignificant decrease was observed in overall DE (primary or secondary diagnoses, harmful or nonharmful) in the post- compared with preimplementation period (16.5 vs. 20.9, OR 0.75 [95% CI 0.43-1.29], p=0.29), or a 25 percent relative risk reduction in overall DE.¹ In postimplementation analysis, researchers found that relative risk reductions in DE were greater in patients in the high-risk cohort. For example, for overall DE, the adjusted OR was 0.66 in the high-risk cohort (95% CI 0.39-1.11, p=0.11) vs. 0.86 in the low-risk cohort (95% CI 0.32-2.31, p=0.76), p=0.64 for the interaction term for effect modification.¹

Overall, researchers found their interventions collectively resulted in trends toward clinically important improvement, particularly for the high-risk cohorts.¹ They noted that because of COVID-19, their implementation was not as robust as they had hoped. However, they found the DTO, as well as the interactive implementation materials (training workshops, interactive newsletters), were well received by clinicians. Researchers recommended that future evaluations include concurrent (e.g., cluster-randomized) trials to reduce confounding by temporal trends due to factors such as the COVID-19 pandemic.¹

To date, this PSLL’s work has resulted in at least 8 peer-reviewed journal publications, with 11 citations in other publications. The team has also been working to disseminate the structured care review process and interventions as part of several AHRQ-funded efforts (UPSIDE, ADEPT).

Publications

2023

• Dalal AK, et al. Identifying and classifying diagnostic errors in acute care across hospitals: Early lessons from the Utility of Predictive Systems in Diagnostic Errors (UPSIDE) study. J Hosp Med 2023;10.1002/jhm.13136.
• Garber A, et al. Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach. JAMIA Open 2023;6(2):ooad031. Published 2023 May 10.
• Schnock KO, et al. Providers' and Patients' Perspectives on Diagnostic Errors in the Acute Care Setting. Jt Comm J Qual Patient Saf 2023;49(2):89-97.

2022

• Malik MA, et al. A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts. Diagnosis (Berl) 2022;9(4):446-457.

2021

• Griffin JA, et al. Analyzing diagnostic errors in the acute setting: a process-driven approach. Diagnosis (Berl) 2021;9(1):77-88.

2020

• Garber A, et al. design and development of a diagnostic time-out to address diagnostic errors in acute care. J Hosp Med 2020. Abstract 386.
• Malik M, et al. Identifying and assessing diagnostic error in acute care: is the electronic health record telling us something? J Hosp Med 2020. Abstract 178.

2019

• Dalal AK, et al. Addressing diagnostic errors proactively using e-triggers to mitigate harm during inpatient episodes of care. J Hosp Med 2019. Abstract 210.

Website

• PSLL 2.0: Improving the Safety of Diagnosis and Therapy in the Inpatient Setting. Harvard University.   

References

1. Bates DW. Final Report: Improving the Safety of Diagnosis and Therapy in the Inpatient Setting. Boston, MA: Harvard Medical School; Brigham and Women's Hospital; Northeastern University; 2023.
2. Bates DW. Research Plan: Improving the Safety of Diagnosis and Therapy in the Inpatient Setting. Boston, MA: Brigham and Women's Hospital; 2018.
3. Griffin JA, et al. Analyzing diagnostic errors in the acute setting: a process-driven approach. Diagnosis (Berl) 2021 Aug 23;9(1):77-88.
4. Malik MA, et al. A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts. Diagnosis (Berl) 2022 Aug 22;9(4):446-457.