Cochlear Implants and Voice Emotion Recognition: A conversation with Dr. Monita Chatterjee
Friday, October 30, 2015
Cochlear implants have helped many individuals with hearing loss understand speech and other sounds since the U.S. Food and Drug Administration (FDA) first approved cochlear implant devices for adults in 1985 and for children in 1990. Monita Chatterjee, Ph.D., Director of the Auditory Prostheses and Perception Laboratory at Boys Town National Research Hospital was recently interviewed for a weekly podcast by Science, one of the premiere scientific journals in the United States.
“Although the technology is constantly evolving, individuals wearing cochlear implants still have difficulty perceiving changes in pitch or frequency of a speaker's voice, which makes it hard to recognize conveyed emotions or the speaker's mood," said Dr. Chatterjee.
One aspect of her lab's NIH-funded research focuses on the study of how well children with cochlear implants can convey and detect voice emotion with their devices.
“With children who have hearing loss and are listening through a cochlear implant, we wonder if their social development relates to their ability to recognize or communicate voice emotion," said Dr. Chatterjee. “There is some published work suggesting that cochlear-implanted children have self-perceived quality of life is related to their voice emotion identification performance. Research is limited and no definite answers are available yet."
In a recent study published in Hearing Research, Dr. Chatterjee's team at Boys Town Hospital and her collaborators at Johns Hopkins University compared voice emotion recognition by individuals with cochlear implants and normal hearing listeners. Using simple emotion-neutral sentences spoken in five different emotions – happy, sad, angry, scared and neutral – they asked listeners to identify the emotion associated with each utterance.
The cochlear implanted children showed large variations in performance but on average performed about as well as adults with cochlear implants. The key difference between these two groups were the cochlear-implanted adults who had hearing at birth and had learned spoken language normally through acoustic hearing, and the children with cochlear implants who had very little or no exposure to sounds at birth and had learned spoken language through the implant itself. The two groups showed similar performance underscores showing the benefit received by the children from their devices. Both groups of implant users, however, performed significantly worse than normal hearing children and adults.
In the same study, children and adults with normal hearing listened to the same sentences but processed digitally through a noise vocoder, which simulated information transmitted by a cochlear implant. Normal hearing listeners' performance declined as sounds were progressively degraded, with the children showing much poorer emotion recognition than adults in the degraded condition.
Results showed normal hearing children who were younger, struggled significantly more with the simulated cochlear implanted speech than older children, even though they all did very well with the natural speech sounds. The children had never heard vocoded speech before and had no prior experience with distorted speech. On the other hand, the children with cochlear implants had long periods of experience with their devices and clearly benefited from it.
These tests show how important it is for the brain to be accustomed to the input sound, especially for younger children. Brains of young children are still developing and the auditory and language systems continue to develop well into the adolescent years. Dr. Chatterjee and her team speculate that their work has implications for peer-to-peer and other social communication, especially at a younger age, which may play an important role in quality of life and social development.
“Hearing with cochlear implants or listening to degraded sounds takes a lot of cognitive effort, and there's more and more interest in the area as we realize the reason cochlear implants work, is that the brain is amazing and does a lot of reconstruction and repair from incoming degraded signals," said Dr. Chatterjee. “The results we found show a huge variation in the performance by these children and adults with cochlear implants – some children with cochlear implants did quite poorly although others did really well in voice emotion recognition, and it really shows how far we have yet to go in cochlear implant research."
While the success of cochlear implants and the availability of the device (children as young as 12 months of age can receive cochlear implants) is a miracle in itself for the medical world, the mission now is to improve the technology in order to optimize hearing and speech communication for patients. In ongoing analyses of children's speech, Dr. Chatterjee's team is finding that children with cochlear implants make smaller acoustic contrasts between “happy" and “sad" emotions than their normal hearing peers. Dr. Chatterjee and her team recently presented their research at the 169th meeting of the Acoustical Society of America in Pittsburgh, PA.
“There's been a lot of discussion in the medical world and it seems like the ability to transmit high-quality voice pitch information to the listener will be part of the next big leap in cochlear implant research," said Dr. Chatterjee. “Understanding more about how the brain processes voice pitch information transmitted by cochlear implants is a key step and we hope by sharing our knowledge and continuing to learn more from our own and other labs' research, we can contribute to the global effort in this area."
The Cochlear Implant Center at Boys Town National Research Hospital has provided cochlear implants to more than 500 children and adults with moderate to profound hearing loss and has more than 600 patients from across the country.
Listen to Dr. Chatterjee's full podcast interview on Sciencemag.org, published on June 12, 2015.
