BlogEmployee StoriesEngineeringInnovation

Animating the ‘Face’ of Reflexion Health: How We Brought VERA to Life

By August 27, 2018 August 26th, 2019 No Comments

At Reflexion Health, we’re proud to be a leader in virtual physical therapy rehabilitation solutions. And we couldn’t do it without our FDA-cleared Virtual Exercise Rehabilitation Assistant (VERA®) platform — or Vera, as we call her.

A lifelike PT assistant who walks patients through the form and pace for their exercises, Vera is the centerpiece of our virtual PT platform. Engaging and responsive, she’s one of the reasons why researchers from a major medical center found our platform to offer “the advantage of cost savings, convenience, at-home monitoring, and coordination of care, all of which are geared to improve adherence and overall patient satisfaction.”1

But Vera wouldn’t exist without the hard work of our world-class development team. In this series, we’re lifting the veil on these essential but behind-the-scenes members of our virtual PT team. In this post, Senior Software Engineer Michael Graessle discusses the unique challenges that came along with our goal of providing Vera with world-class animation — and how those challenges were overcome to bring Vera to life.

Animating the ‘Face’ of Reflexion Health

We needed a way to easily and effectively create animations for Vera’s face so she could talk along with the audio that we’d recorded. There are hundreds of things that Vera can say, so we needed to find a process to sync up audio with the face movements without having to touch up the animation or create it by hand — an expensive and time-consuming process

Hand animation is a wonderful art, but it wouldn’t have been the right approach for Vera. The face is a complex thing to animate by hand, or any other way. Consider all the parts that need to be animated on the face — the lips, tongue, eyes, eyebrows, cheeks, and so on — and then consider that each of these features needs to have unique, lifelike qualifies.

Since they are automated, we only needed to create one piece of animation that contained the necessary phonemes, then match each up with a corresponding facial expression from Vera. In this way, the software learned the definition of what Vera’s facial expression would look like when she said a diverse range of sounds.

Once hit upon, our process for creating Vera’s animation — and especially her facial expressions — made it possible for Vera to come to life as an engaging avatar that patients find interactive and even fun, without bogging down our small startup organization with unrealistic manpower hours.

With this innovation in place, we now have the ability to process a large number of facial animations for Vera very rapidly, allowing us to add new things Vera can say in the future as well. And we’re also free to move on to other challenges — which can be an unending list for healthcare tech startups (and the DevOps engineers who work for them).

Michael Graessle is a Senior Software Engineer for Reflexion Health.

Sources

1 Chughtai et al. The Role of Virtual Rehabilitation in Total and Unicompartmental Knee Arthroplasty. J Knee Surg. 2018 Mar 16. doi: 10.1055/s-0038-1637018. [Epub ahead of print]