February 2019

Reinforcement Learning Expedites Tuning of Robotic Prostheses

Photo courtesy of NC State

When an individual is fitted with a robotic prosthetic knee, a practitioner works with that person to modify parameters to specifically accommodate their gait, a process that can take hours. Now, a multidisciplinary team of researchers from Arizona State University (ASU), Tempe; North Carolina State University (NC State), Raleigh; and the University of North Carolina at Chapel Hill (UNC Chapel Hill) have developed an intelligent system for tuning powered prosthetic knees, allowing users to walk comfortably with the device in minutes rather than hours. This is the first system to rely solely on reinforcement learning to tune a robotic prosthesis in the published literature IEEE Transactions on Cybernetics.

The algorithm, which essentially teaches a prosthetic device to adapt to a user’s normal walking gait using data collected from sensors in the device and the person’s natural walking pattern, was developed by Jennie Si, PhD, a professor in the School of Electrical, Computer and Energy Engineering in ASU’s Ira A. Fulton Schools of Engineering. The program uses reinforcement learning to modify 12 control parameters that address prosthesis dynamics, such as joint stiffness, throughout the gait cycle. With this system, a person using a powered prosthetic knee can walk on a level surface in about 10 minutes, “but in principle, we could also develop reinforcement learning controllers for situations such as ascending or descending stairs,” said Si.

Si has been working on reinforcement learning from the dynamic system control perspective to account for sensor noise, interference from the environment, and the demand of system safety and stability. While the work is currently done in a controlled, clinical setting, one goal would be to develop a wireless version of the system, which would allow users to continue fine-tuning the powered prosthesis parameters when being used in real-world environments.

Si and her colleagues hope to make the process even more efficient, for example, by “identifying combinations of parameters that are more or less likely to succeed and training the model to focus first on the most promising parameter settings,” said He (Helen) Huang, PhD, a professor in the Joint Department of Biomedical Engineering at NC State and UNC. Huang and Si are co-authors on a paper about the work, which was published in the journal IEEE Transactions on Cybernetics.

“The prosthesis tuning goal in this study is to meet normative knee motion in walking,” said Huang. “We did not consider other gait performance, such as gait symmetry, or the user’s preference. As another example, we can use our tuning method to fine-tune the device outside of clinics and labs to make the system adaptive over time with the user’s need. However, we need to ensure the safety in real-world use since errors in control might lead to stumbling and falls. Additional testing is needed to show safety.”

If the system proves to be effective and enters widespread use, it could reduce costs for prosthetic knee users by limiting the number of clinical visits needed to work with practitioners on their gait.

Leave a Reply

Your email address will not be published.

This site uses Akismet to reduce spam. Learn how your comment data is processed.