Final Exam: A Feel for Music

With ‘Haptic Hero,’ a senior design team helps make gaming more fun for young people who are visually impaired.

“Meaningful making.” It’s a guiding principle that inspires Stevens senior Richard He, and is just one of the many things he’s learned from his longtime mentor Ian Matty, co-founder and director of the robust Mountain Lakes (New Jersey) Public Library Makerspace.

I was working last summer with Mr. Matty at St. Joseph’s School for the Blind in Jersey City, New Jersey, helping to run a technology week, and just talking to the students I realized there is a serious lack of options for them in the recreation space.
Richard He

It was with that idea in mind that He recruited five like-minded seniors to create — as their senior design project — a fun, affordable and easy-to- learn video game for young people who are visually impaired, a project for which Matty has served as an enthusiastic outside advisor [see sidebar].

The six members of the Haptic Hero senior design team pose for a photo.From left: Haptic Hero team members Hasumi Tanemori, Kimberly Tsang, Richard He, Paul Leible, Emanuel Diaz and Sebastian Almonte.

A mechanical engineering major, He developed an abiding passion for 3D printing while he was still in high school and was deeply involved with Matty’s Makerspace. The Stevens senior loves to teach 3D printing — including to the kids at St. Joseph’s — so it’s integral to the project and to the goal of creating two working prototypes for the game.

The idea of creating something fun for students who are visually impaired resonated immediately with the Stevens students He approached to join the team: Sebastian Almonte, Emanuel Diaz and Paul Leible, all mechanical engineering majors, and Hasumi Tanemori and Kimberly Tsang, both computer science majors with invaluable programming skills.

They dubbed their game Haptic Hero, modeled after Guitar Hero, a popular 2005 video game that rewards players for hitting the right notes as they attempt to play along to a song. The students have replaced visual cues with haptics — tactile information derived from sound and vibration. And they’ve spent countless hours brainstorming, researching and designing, as well as ordering and testing inexpensive electronic components.

A graphic offers an inside view of a key component of the Haptic Hero game.Here’s an “exploded” view of the prototype’s haptic module, which cues notes to the player and receives their button inputs.

But they’ve also spent a lot of time consulting with their intended audience. The team traveled to Princeton, New Jersey, in October for Island 2023, a conference on disability and STEM, and later that month they went to St. Joseph’s to brainstorm with students and get feedback on a 3D-printed control pad they had developed for the game.

“One of the first things we learned was that this is not about us trying to imagine what a video game for visually impaired players might look like; it’s about listening to our potential customers and finding out what they actually want,” says He.

The team members are deeply committed to the project and they also work extremely well together, says their advisor Frank Fisher, professor and associate dean for undergraduate studies in the Stevens School of Engineering and Science, who meets with them weekly.

“This is a student-generated idea, and I think it has huge potential,” Fisher says. “Imagine going through the process of creating something that can change people’s lives.”

Universal Design

A close-up view of the Haptic Hero prototype, shaped like a real electric guitar. The face and neck of the purple guitar are open the electrical wiring and internal speaker are exposed. In the center of the guitar is an iPhone. Photo: Michael Marquand

“Accessibility should not be an afterthought,” says Richard He ’24, a mechanical engineering major and leader of the Haptic Hero senior design project team. “That’s one of the things we’ve learned working on this project. You have to think about accessibility from the very beginning.”

He and his team have relied heavily on the counsel of Ian Matty, co-founder and director of the Mountain Lakes (New Jersey) Public Library Makerspace and a longtime proponent of universal design. “It’s just about designing better for everybody,” says Matty, who has worked with students on a long list of projects using technology to promote accessibility, including using 3D printing to make books more richly tactile.

“One thing we know is that all kids love games, so creating a game [like Haptic Hero] in which a visually impaired kid can play on a level playing field with a sighted kid is just a phenomenal way to help make a more inclusive world,” Matty says.