Commercial AI tool with moderate success in predicting hospitalization-related kidney injury

Hospital-acquired acute kidney injury (HA-AKI) is a common complication in hospitalized patients that can lead to chronic kidney disease and is associated with longer hospital stays, higher health care costs, and increased mortality. Given these negative consequences, prevention of HA-AKI may improve outcomes for hospitalized patients. However, it is difficult to anticipate the onset of HA-AKI due to the large number of contributing factors involved.

Researchers at Mass General Brigham Digital tested a commercial machine learning tool, the Epic Risk of HA-AKI Predictive Model, and found that it was moderately successful in predicting HA-AKI risk in recorded patient data . The study found lower performance than recorded by Epic Systems Corporation’s internal validation, highlighting the importance of validating AI models before clinical implementation.

The Epic model works by assessing adult hospitalized patient encounters for risk of HA-AKI, marked by predefined increases in serum creatinine levels. After training the model using data from MGB hospitals, the researchers tested it on data from nearly 40,000 hospital stays over a five-month period between August 2022 and January 2023. The data collection was extensive and contained many points collected from patient encounters, including information such as patient demographics, comorbidities, primary diagnoses, serum creatinine levels, and length of hospital stay. hospital. Two analyzes were performed to examine the performance of the models at the encounter level and at the prediction level.

The investigators observed that the tool was more reliable when evaluating patients at lower risk of HA-AKI. Although the model could confidently identify which low-risk patients would not develop HA-AKI, it had difficulty predicting when higher-risk patients might develop HA-AKI. Results also varied depending on the stage of HA-AKI assessed: predictions were more effective for stage 1 HA-AKI than for more severe cases.

The authors concluded overall that the implementation could result in high false positive rates and called for further study of the clinical impact of the tool.

“We found that the Epic predictive model was better at excluding low-risk patients than at identifying high-risk patients,” said the study’s lead author, Sayon Dutta, MD, MPH, of the clinical informatics team at Mass General Brigham Digital and an emergency room physician. at Massachusetts General Hospital. “Identifying HA-AKI risk with predictive models could help support clinical decisions, such as warning providers not to order nephrotoxic medications, but further study is needed before clinical implementation.”

The study is published in the journal NEJM AI.