New Sony Partnership Makes TVs Easier to Hear

It’s hard to overstate the importance of sound in entertainment. Whether we’re binge watching our favorite TV show or creating a playlist of songs to work out to at the gym, what we see and hear in the entertainment we choose shapes our cultural experience and may even influence our identity.

But for the approximately 48 million Americans living with hearing loss, listening to music or watching TV can be more of an exercise in frustration than relaxation. The struggle to hear lyrics, make out dialogue, or isolate discrete parts of a recording can chip away at the pleasures of recreational listening. And as those with hearing loss know, it’s a problem not solved by simply turning up the volume.

“Hearing loss is more than just not hearing soft sounds,” said Larry Guterman, Chief Customer Experience Officer of hearing technology app SonicCloud. “People with hearing loss have a lower tolerance for loud sounds and for different frequencies.”

For example, some people with hearing loss may hear consonants but not vowels, while others struggle to understand high pitched voices or distinguish a speaker from background noise. The range of hearing loss is so wide that audiologists typically conduct a series of specialized hearing tests on patients before prescribing a hearing aid.

But in 2017, Congress passed a bill legalizing the sale of FDA-approved, over the counter (OTC) hearing aids without a prescription. With the new law came a surge of interest from consumer electronic giants like Apple and Bose, who saw an opportunity to cater to a previously underserved market: the hearing loss community. Just last year, Apple introduced a new personalized amplification feature in the AirPods Pro.

“For decades, the hearing aid industry was regulated, and to get a hearing aid worth its weight in salt you had to get a prescription,” said Sachin Khanna, CEO at Sonitum Inc., the makers of SonicCloud. “This bill tries to address the eighty percent of people who could benefit from hearing aids but don’t wear them.” Denial and stigma account for some the 80%, according to Khanna, but cost is still the big one, and medical insurance typically does not cover the cost of hearing aids.

“Deregulation makes it easier for people to get into the journey of better hearing,” Khanna said. “If we make products that are over the counter, hopefully people will feel like it’s easier to jump into the hearing world as opposed to going to the doctor and having this clinical experience.”

The FDA still hasn’t issued formal rules on OTC hearing aids, but that hasn’t stopped companies from working towards a deregulated future.

Sony was actively looking to develop new products for the hearing loss community in 2018, when they discovered SonicCloud at an assistive technology convention. “They were intrigued by us,” Guterman said of Sony. “Their interest went beyond paying lip service to accessibility—they wanted technology that would dial in not just decent but great and precise speech understanding. Our goals were aligned in that regard.”

The companies formed a partnership and SonicCloud began building a prototype a for new hearing assistance product aimed at customers with hearing loss. The new product would be driven by SonicCloud technology and would available for direct purchase by consumers, just like the over-the-counter hearing aids of the future.

“When you think of televisions, the first brand that comes to mind is Sony,” Khanna said. “Their flagship product is the television. So that’s where we started.”

What is SonicCloud Technology?

Guterman, a former Hollywood film director (Cats and Dogs) with a degree in physics from Harvard, said he never envisioned starting a tech company. But when his own noise-induced hearing loss became so severe that he could no longer confidently take phone calls, he began searching for solutions.

“I was working in this business where it was really important to talk on the phone, but I was getting terrible feedback with my hearing aids and I didn’t have good speech discrimination,” he said. “So, I teamed up with engineers and Jody Winzelberg (former chief of audiology at Stanford Children’s Health) and we said, ‘why not take the functionality of the $7,000 hearing aid and stick it on the mini super-computer we’re all carrying around in our pocket?’ (i.e., smart phones)”

They started by creating a hearing profile for Guterman on a computer.

“We had access to parameters audiologists typically weren’t given access to because it was our own software,” Guterman said.  “We were able to measure down to sub-1 decibel, we measured discomfort levels to establish my dynamic range and we were able to manipulate additional compression parameters that aren’t normally part of the standard fitting software.”

Once the prototype was ready, Guterman took a call from his wife wearing Apple ear buds instead of his usual hearing aids. As they talked, his wife’s voice was filtered through the software containing Guterman’s hearing profile. The conversation left his wife “bawling,” he said.

“She said to me, ‘you don’t understand, we’ve been married 15 years and you ask me every thirty seconds to repeat myself, and we just talked for thirty minutes and you didn’t ask me once,’” he recalled. “So, after that, we knew we had the prototype, and now we had to figure out how to turn it into a business.”

They decided to build an app that could be downloaded to a smart phone or computer. Users would take a test to create a hearing profile that would be used to customize their listening experience on everything from phone calls and web meetings to listening to music and watching videos.

“The idea was to try to make it a fun, engaging test so it wouldn’t be a drag,” Guterman said. “I brought in some of my experience from the animation movie world. I wanted it to be not too medical and clinical, but more like a game.”

Fine tuning the listening experience in real time

“You’re getting a personalized sound, but you can also make adjustments,” Guterman said. “You have the ability to adjust the left ear and the right ear for richness, vowels, boost, comfort and fullness. This (feature) represents about 150 audio parameters under the hood.”

While the adjustment tool may look similar to a stereo equalizer, Guterman said SonicCloud’s capabilities are far more advanced.

“A regular equalizer has a very narrow volume range and is just equalization,” he said. “We’re dealing with multiple other parameters affecting the character of the sound to sculpt it for your hearing loss.”

For example, if a user is talking on the phone and wants to emphasize the sharpness of the consonants in the other person’s voice, the app allows them to do so—and to save the profile for future phone calls.

“You can customize the profile for whoever you’re speaking with or the environment,” Guterman said. “If there’s a noisy background, often the low frequencies will be where noise is coming from, so you want to reduce the richness to help direct the sound toward the person who’s speaking.”

The Immersive Wearable Speaker

In 2019, Sony unveiled the SRS-WS1 Immersive Wearable Speaker, a U-shaped device that rests on the user’s shoulders and “immerses” them in personalized sound and vibrations as they watch TV.

“The way it works in tandem with the SonicCloud technology is the SonicCloud hardware piece is connected to the Sony TV,” Guterman said. “The user’s personalized hearing profile is beamed to the neck speaker, resulting in personalized sound.”

The user can also select other SonicCloud profiles from an app on their phone and make real time tuning adjustments while they watch television. “You might want different  profiles for different content, to optimize how you hear them,” he said.

“The change in the marketplace and the change on Capitol Hill all came together,” Khanna said of the partnership with Sony. “Television is just the start.”

Guterman agrees the possibilities for accessibility products are endless. “Our tech is modular, so ultimately it will be able to be used with other Sony hardware beyond the neck speaker,” he said.

As hearing devices become more personalized and widely available, Guterman said he hopes the stigma associated with hearing loss will fade. He noted approximately ten to fifteen percent of current SonicCloud users do not have hearing loss and use the app for listening to music.

“Composers, folks that work at big audio platform companies use it because they like the enhancement they get and they like the control,” he said. “In this world of diversity and inclusivity, hearing loss still seems to be a stigma, but I think that’s starting to change. There’s a lot of transformation happening.”