News, jobs & events in Pittsburgh for our community of tech professionals

The ultrasound phased array allows users to feel mouth sensations in the metaverse without an added device.

That’s right — with the help of students and faculty at the Future Interfaces Group within the university’s Human-Computer Interaction Institute, anyone hanging out in a virtual reality space might soon be able to drink from a water fountain, brush their teeth or even feel a bug crawling across the mouth.

It’s all thanks to an ultrasound phased array, technology that is able to focus air-borne acoustic energy onto the lips, mouth, teeth and tongue to create sensations like taps and continuous vibrations without putting a device over or in the mouth. The half moon-shaped array can be attached to the bottom of existing virtual reality headsets in a way that allows ultrasonic transducers to focus beams of acoustic energy onto the mouth.

The FIG Lab, as it’s called, isn’t the first to look into VR applications for haptics, a field that studies the sense of touch and how it can be recreated using technology. Even mouth haptics aren’t new in the metaverse. But second-year CMU Robotics Institute Ph.D. student Vivian Shen told Technical.ly that the FIG Lab is the first to leverage ultrasound phased arrays for integration into VR headsets for mouth haptics.

“Right now, really the only commercial haptics path is basically vibration. So your controllers will vibrate when you touch something or vibrate when you hit something or vibrate when you feel something,” Shen said. “And that’s really just not what we feel in our everyday lives.”

While ultrasound phased arrays can’t simulate more detailed mouth sensations that cover a large amount of surface area (like kissing, for example), they don’t require any additional technology to be attached over or in the mouth, lowering the barrier to entry for the user and making the overall experience more comfortable. But that doesn’t mean there’s not a place for those approaches too, Shen said — they just haven’t been widely commercially adopted as of now.

“All sorts of haptics research is necessary if we want to make virtual reality a more immersive and realistic experience,” she said.

That work, encompassed in a recent paper by Shen, CMU post-doctoral fellow Craig Shultz and associate professor Chris Harrison, earned the research team the Best Paper Award at the Association for Computing Machinery’s Conference on Human Factors in Computing Systems, held earlier this month.

A user brushes their teeth in the metaverse using the ultrasound phased array developed by CMU’s FIG Lab. (Courtesy photo)

Creating mouth sensations — or any tactile feeling, for that matter — in the metaverse is fun for immersive gaming experiences, but it also has the potential to enhance important simulations, Shen said. For mouth haptics in particular, one example of an area Shen foresees this tech benefitting down the road is dental simulations for students or trainees in the field. The device, she said, could teach them how to clean or treat patients without requiring actual human patients.

“Our device isn’t quite expressive enough to really emulate a dental cleaning,” Shen said. “But if we are able to enable those types of haptics, then training simulations will be so much more realistic and so much more useful.”

For now, Shen said the lab will be continuing to improve the ultrasound board they’ve been working on and inviting users to come to the lab and try it out for feedback. Moving forward, she and the other researchers have started looking into other mid-air haptics, creating tactile sensation for something that doesn’t actually have to touch the user, like a fan.

“We think of this as an interesting way to be able to have people feel stuff without having to put on a whole suit or something, and having stuff touching their skin, which is a lot more restrictive.” Shen said of the lab’s work in the field.

And while mention of the metaverse can spark eye rolls these days on some corners of the internet, Shen thinks innovation in the field is important because the technology will certainly have a place in the the future.

“I disagree with the idea of the metaverse becoming this thing that like everyone has to get on and that we’re going to be spending hours and hours and hours of our everyday [life on it],” she said. But its potential for enhancing remote trainings and simulations and the opportunities it creates to bring people together in new ways who might be physically apart are worth exploring. She points to how social media changed the world by creating new ways of connecting people across the globe.

“Now you can message anyone around the world in seconds, and you can keep in contact with friends that have left the country, and it’s really connected the world,” she said. “And I think similarly, when VR becomes more commonplace, it will feed that connection.”

Sign-up for daily news updates from Technical.ly