How Pitt researchers are using AI to help stroke patients

Researchers from Pitt’s School of Health and Rehabilitation Sciences are collaborating to improve how post-stroke rehabilitation is administered with help from an unexpected source — artificial intelligence.

Elizabeth Skidmore, a rehabilitation scientist and professor in the Department of Occupational Therapy, studies ways to improve how clinicians help people experiencing cognitive challenges after a stroke. She is a proponent of a rehabilitation method called strategy training, which shifts a rehabilitation therapist's role from an authoritative instructor to one of a supporting player in a patient-driven process.

“Therapists can look at a situation and come up with a solution in a snap second, but the effective rehabilitation isn’t about the therapist’s wisdom,” Skidmore said. “It's about the client because it's their life, and they are going to go on to live it after their therapy is done.”

In other words, her training gives stroke survivors more control over their recovery by training them to prioritize the tasks that matter to them, develop a plan to execute the activities and practice problem-solving skills.

At present, Skidmore has conducted three randomized, controlled clinical trials to examine the effectiveness of strategy training and aspires to one day run a national multisite trial of strategy training in rehabilitation facilities. But first, her team has to make the training replicable.

“When it comes to a wide-scale implementation of strategy training, we need a way to give therapists feedback on whether the intervention they deliver is consistent with the intervention that we believe is associated with the best possible outcomes,” she said.

This is where AI can help: training the trainers.

One of the most time-consuming and costly elements of her research is evaluating how successfully therapists implement her training, Skidmore said. She wondered whether computers could assist.

To help turn observations into data, Skidmore employs fidelity raters — licensed occupational therapists and occupational graduate students — to watch recorded rehabilitation sessions and complete a checklist as the clinician uses appropriate cueing strategies, the hallmark of strategy training. Examples include asking open-ended questions, such as “What do you think about …?” or using guiding statements, such as “Let’s consider the options.” This is in contrast to direct skill training — instructions like, “Tie your shoes like this” or “Pay attention to the loose gravel on the walkway.”

“The biggest thing we’ve learned so far is that most of our therapists are well-trained and have developed well-honed instincts, but they’re not always conscious of how they provide training. Giving feedback based on their recorded sessions helps them execute strategy training with greater consistency,” said Skidmore.

“On average the therapists we’ve evaluated are using guided cues 5% of the time. Our studies suggest increasing guided cues to 40% or 50% of the time can significantly improve client outcomes. It just requires training therapists to monitor and change their habits,” she added.

The first steps

Skidmore didn’t have to look far to find help for her project. In February 2022, Yanshan Wang, vice chair of research and assistant professor in the Department of Health Information Management, and Leming Zhou, an associate professor in the same department, worked with Skidmore and her team as principal investigators and began developing an algorithm-based technology to produce an evaluation analysis in just minutes, with funding support from the University of Pittsburgh Clinical and Translational Science Institute’s Quantitative Methodologies Pilot Program.

AI is already widely used in health care settings but is often limited to one dimension — natural language processing for classifying clinical documentation or machine learning for future outcome prediction, for example. Wang and Zhou’s approach is multimodal: They aim to align computer vision, natural language processing and machine learning — a groundbreaking advancement in AI applications.

“AI is not magic,” said Wang, “it can’t create something from what we don’t know and do something that a human has no idea about. Think of it more like augmented AI — it can help us make workflow and fidelity assessment more consistent and efficient.”

Wang and Zhou began their research by closely observing how the fidelity raters annotated the recordings and then considering how they might automate the procedure.

Step one: Create a gold standard dataset to develop an algorithm using transcripts from video-recorded rehabilitation sessions. Step two: Test its accuracy against a trained human fidelity rater.

They noted their progress in a paper to be published at the AMIA 2023 Informatics Summit, and the results are promising. Hunter Osterhoudt, a graduate student in the Department of Computer Science, is the first author on the paper and will present this work at the conference.

Zhou and Wang said although there is room for improvement, their verbal processing results met industry reliability standards and responded to the challenges inherent in Skidmore’s project.

Forging innovative research and being the first in the field takes dedication and patience. Looking ahead, the team plans to next integrate computer vision, training the algorithm to recognize different types of physical gestures used in the rehabilitation procedure.

Skidmore estimates the automation project will be in development for a few years before it’s ready and available for commercialization and widespread use.

But the wait is worth it, she said. “We’ll do what it takes to study and improve care.”

— Nichole Faina, imagery by Getty