Wearable cameras enable AI to detect medication errors

A team of researchers claims to have developed the first wearable camera system that, with the help of artificial intelligence, detects potential errors in medication administration.

In a test whose results were published on October 22 in npj Digital Medicinethe video system recognized and identified, with high efficiency, which medications were being collected in busy clinical environments. The AI ​​achieved 99.6% sensitivity and 98.8% specificity in detecting vial exchange errors.

The system could become an essential protection, especially in operating rooms, intensive care units and emergency medicine settings, said co-senior author Dr. Kelly Michaelsen, assistant professor of anesthesiology and in Pain Medicine at the University of Washington School of Medicine.

“The idea of ​​being able to help patients in real time or prevent a medication error before it happens is very powerful,” she said. “We can hope for 100% performance, but even humans can’t achieve that. In a survey of more than 100 anesthesiologists, the majority wanted the system to be more than 95% accurate, which is a goal that we have achieved.”

Medication administration errors are the most frequently reported critical incidents in anesthesia and the most common cause of serious medical errors in intensive care. Overall, it is estimated that 5-10% of all medications administered are associated with errors.

Adverse events associated with injectable medications are estimated to affect 1.2 million patients each year, at a cost of $5.1 billion.

Syringe and vial exchange errors most commonly occur during intravenous injections in which a clinician must transfer the medication from the vial to the syringe to the patient. About 20% of errors are substitution errors in which the wrong vial is selected or a syringe is mislabeled. Another 20% of errors occur when the medication is correctly labeled but administered in error.

Safety measures, such as a barcode system that quickly reads and confirms the contents of a bottle, are in place to guard against such accidents. But practitioners can sometimes forget this check during high-stress situations because it’s an extra step in their workflow.

The researchers’ goal was to create a deep learning model that, combined with a GoPro camera, was sophisticated enough to recognize the contents of cylindrical vials and syringes and to issue an appropriate warning before the drug entered the patient.

Training the model took months. Investigators collected 4K video of 418 drug draws performed by 13 anesthesiology providers in operating rooms where configurations and lighting varied. The video shows clinicians handling vials and syringes of selected medications. These video clips were then recorded and the contents of the syringes and vials noted to train the model to recognize the contents and containers.

The video system does not directly read the text on each vial, but looks for other visual clues: size and shape of the vial and syringe, color of the vial cap, print size of the label.

“It was particularly difficult, because the person in the operating room is holding a syringe and a vial, and you don’t see any of these objects completely. Some of the letters (on the syringe and vial) are covered by the hands. And the hands” They are moving quickly. They get the job done. They’re not posing for the camera,” said Shyam Gollakota, co-author of the paper and professor at UW’s Paul G. Allen School of Computer Science & Engineering.

Additionally, the computer model had to be trained to focus on the drugs in the foreground of the frame and ignore the vials and syringes in the background.

“AI does all of this: detecting the specific syringe that the healthcare provider is picking up, and not detecting a syringe sitting on the table,” Gollakota said.

This work shows that AI and deep learning have the potential to improve safety and efficiency in a number of healthcare practices. Researchers are just beginning to explore its potential, Michaelsen said.

The study also included researchers from Carnegie Mellon University and Makerere University in Uganda. The Toyota Research Institute built and tested the system.