Tuning Music to Fine-Tune Human Movement: Using Biofeedback to Improve Movement Mechanics in Athletes and Older Adults
In Dr. Antonia Zaferiou’s laboratory, engineering embeds art
This month, Stevens Institute of Technology will host its fifth annual Introduce a Girl to Engineering! Day event at Brensinger Elementary School in Jersey City. Throughout February, we’re also sharing the stories of some of our female faculty members and students, and how they are critical to our mission to inspire, nurture, and educate leaders in tomorrow's technology-centric environment while contributing to the solution of the most challenging problems of our time.
These notable female researchers are fundamental contributors to the achievements that earned Stevens the Bronze Award in the American Society for Engineering Education’s inaugural Diversity Recognition Program for helping us make “significant, measurable progress in increasing the diversity, inclusion, and degree attainment outcomes.”
Engineering and art meet at the intersections of movement training, rehabilitation engineering, dance, and music—a convergence that drives Stevens Institute of Technology’s Assistant Professor Antonia Zaferiou, Ph.D.
Zaferiou, a professor in the Department of Biomedical Engineering, is using sonification—a form of biofeedback in which participants create sound through their movement—to train movement strategies during everyday mobility tasks and specialized athletic maneuvers.
Zaferiou focuses on understanding the neuromuscular control of human movement across the movement performance spectrum—from elite dancers performing turns, to professional pitchers, to clinical populations who struggle to move. Using biofeedback, athletes can fine-tune their movements, while older adults who have fallen can learn to better control their balance during tasks such as turning while walking in real-world conditions.
Her research in this field recently earned her the prestigious National Science Foundation (NSF) CAREER Award.
Training movement through sonification
“My lab focuses on understanding how people control their movements and learning how to train people to move in specific ways using sound or music as biofeedback,” said Zaferiou. “To affect change in movement, sonification presents an exciting opportunity to do so while leveraging the connections people have between movement and soundscapes, or music.”
Sonification is an emerging type of biofeedback, which is a technique used in clinical settings to help participants learn to control aspects of their body’s functions. During biofeedback, a participant is connected to noninvasive sensors, which are connected to a computer, which outputs information about the body, such as heart rate. The participant uses this feedback to monitor or make changes in their behavior, which are then reflected in the computer’s output. This is an interactive learning process for participants that can be very helpful. Clinical applications are varied, including reduction in chronic pain or anxiety.
In Zaferiou’s lab, participants wear small sensors as well as interact with sensors installed in the ground that measure variables such as muscle activity, ground reaction forces, or more. Through “sonified” biofeedback, participants can create sound with their movements, and they can manipulate and learn from this sound by fine-tuning their movements.
The mechanics of turns necessary to daily living and athletic maneuvers
“Sonified biofeedback conveys biological signals through sound, which presents a novel modality to train movement during turns,” said Zaferiou. “Turns are essential to daily mobility, but goals to turn the body often conflict with maintaining balance. While the success of an athlete often relies on the ability to turn with ease, people with sensorimotor challenges have difficulty turning.”
Engineers of bipedal (two-legged) robots face challenges in designing machines capable of turning while maintaining balance, something most people take for granted in their daily activities. Some of us may remember playing with a soft robotic dog as children, which barked as it shifted from a standing to sitting position and then performed a backflip. This impressive feat was made possible partly because the toy dog had four legs to land on and widened its stance before pushing off the ground. By enlarging the area of the contact with the ground, the toy dog made the task of balancing easier as it pushed on the ground. When it pushed on the ground to take off, it did so in a particular way to apply forces to rotate its body. However, many turns that humans or bipedal robots perform do not have the luxury of a wide base of support during their “initiation” (like take-off) or “termination” phases (like landing).
Movement control becomes more difficult when engineers design a biped robot to, for example, perform a flip while rotating from a frontward to a backward orientation while balancing on two legs. The mechanical objectives of turning that need to be managed simultaneously include translation, rotation, and balance. Managing these objectives while turning supported by a small area in contact with the ground is a skill that many dancers and athletes have refined through extensive training. In contrast, those who struggle with mobility may need to learn to perform turns encountered in daily living by learning to manage one objective at a time.
Zaferiou studies the way a human body turns through a systematic and mechanical lens. Complementary to this approach, in collaboration with a sound designer, she has developed a sonification system to help people learn how to better control their balance or movement patterns. Through sonification, she engages dancers and athletes in training, older adults in movement retraining, and even young students who are beginning to develop their interests in STEAM— science, technology, engineering, art, and math. STEAM incorporates art into STEM (the same acronym, without the arts)— to help students, researchers, and professionals develop critical and creative thinking skills to solve the world’s problems holistically.
Refining movement performance
Skilled dancers and athletes learn their art through discipline and professional training. Zaferiou describes sonification as a novel training method that may help them refine their performance through deepening an understanding of the underlying principles of movement.
“There are two branches to our growing research focused on improving and preserving movement mechanics,” said Zaferiou. “One branch focuses on helping people with sensorimotor challenges, and the other branch focuses on how athletes, including dancers, act as model dynamic systems.”
Sonification can help athletes, dancers, and gym-goers who are already engaged in physical activity learn to fine-tune their movements. Many dancers are accustomed to moving their bodies in synchrony with sounds, and many athletes listen to music while training. Listening to music while training may help establish rhythm and motivate focused training. By providing extra sensory information through sonification, dancers and athletes can merge the existing strengths of training with music with a new element of better understanding underlying mechanics in real-time. During sonification, as dancers and athletes listen to the sounds their bodies generate, they can make small adjustments to create the desired sound—which requires them to perfect their movements.
For example, maintaining specific pelvic alignments is essential in ballet. Zaferiou describes how it is possible to monitor the alignment of the pelvis using one wearable sensor. Using wearable sensors and sonification, ballet dancers can increase their awareness of their pelvic alignment as they move freely in space.
Shifting the focus from rehabilitation to preventative approaches
Zaferiou’s recent NSF CAREER award focuses on improving how older adults balance during everyday mobility. The first objective of the grant is to quantify and understand how older adults balance during different types of turns performed inside a lab and outdoors. The second objective is to develop and test if and how sonified biofeedback can improve balance behavior on a person-specific basis. These studies will inform the future development of automated and personalized balance biofeedback.
Her NSF CAREER study was informed and motivated by her dissertation and experience at the Veterans Affairs. During her dissertation research at the University of Southern California, Zaferiou analyzed how dancers complete different types of full body rotations, accounting for different rotation and translation demands—such as single, double, and triple revolutions that are stationary or moving through space. She then applied this approach to better understand how older adults turn in daily life. She worked with Veterans Affairs of Greater Los Angeles to study turning and walking tasks performed by older adults who have fallen.
“I want to meet the clinical need to improve mobility during everyday tasks,” said Zaferiou. “A lot of researchers focus on straight line walking and stationary balance, but I’d like to better understand dynamic balance in real-world contexts. We sway, we are constantly maneuvering and adjusting to interact with our environment. We don’t walk in a straight line all day on even ground.”
We live in a world where we tend to react to injuries, rather than focus on their prevention. While Zaferiou’s work could have important implications for rehabilitation and physical therapy after an injury, such as one caused by a fall, what she is most interested in is developing infrastructure for preventative approaches.
Prevention is the goal of prehabilitation, or “prehab.” Unlike rehabilitative care, which takes place after an injury or decline in ability has already occured, prehab is a proactive approach to prevent such difficulties from occurring to begin with. It can address deficits in strength, stability, range of motion, and joint function, as well as help to decrease pain, prepare for surgery, or reduce the risk of an injury.
“The goal is that people struggling with mobility can be engaged in movement training before further declining,” said Zaferiou. “We can build systems to keep people balanced, instead of waiting until after a fall or serious injury when it may be too late. My lab wants to enable people to preserve or improve their mobility in interactive ways. These interactions can be both ‘one-on-one’ with technology and enveloped within a collaborative and social environment.”
According to a study published by the National Institutes of Health, “physical activity has been consistently associated with enhanced quality of life in older adults.” Self-efficacy, the study reports, is associated with better quality of life; and self-efficacy largely depends on the breadth of movement ability. Zaferiou and her team want to preserve or improve the movement abilities of older adults to help them live better lives.
“Your part in society shifts as you age, and a lot of times there’s fewer and fewer ways for older people to participate in society,” she said. “I foresee movement training in a group setting where people can work together to better prepare themselves to move in real-world conditions.”
The influence of art in engineering
Zaferiou trained in ballet and contemporary dance from the age of four until college, all while learning about the mechanics of the world around her to satisfy her intrinsic curiosity and drive for knowledge. “Growing up, I loved dancing and building things, disassembling them, and rebuilding them,” she said. “My grandfather was an electrical engineer. When I discovered his workshop and workbench, he taught me about conveying my designs with technical drawings and how to build. He remains my daily inspiration.”
In high school, learning physics came easily to Zaferiou, which she related to her first-hand experiences in dance. “I was most fascinated by understanding how things work, especially how movement manifests,” she said. “With my background in dance, kinetics and dynamics made sense to me because I had grown up experiencing and embodying these phenomena.”
It seemed natural to Zaferiou that she study mechanical engineering in college. While her primary goals were within the engineering world, she maintained a practice of dance as a hobby throughout her studies. When she was introduced to the field of biomechanics, she found this to be the perfect opportunity to merge her mechanical engineering knowledge with her interest in movement—specifically, movement of the human body coupled with music.
Zaferiou’s new lab provides more space for broad movement
This spring, Zaferiou is moving into a new lab that is better equipped for her sensor systems and allow room for movement, such as jumps, turns, leaps, and running. The lab includes force sensors in the ground along with optical motion cameras, and wearable inertial measurement units. It will also allow her to host STEAM outreach events within the space, so students can experience and interact with technology to see and hear the forces and muscle activations driving their movements (watch an example here).
The future of movement training
“In the future, I foresee that people with movement disorders will be able to understand how they move and how to interact with technology to improve their movement,” said Zaferiou. “Older adults will be able to engage in immersive technology that they take home, and then they can come into a facility and practice movement in a community-based, interactive training.
“This framework can extend to help anyone from weekend warriors in running groups to elite dancers perfecting their pirouettes. The overarching goal is to have people feel they are in control of their musculoskeletal health and mobility.”
Learn more about biomedical engineering at Stevens: