Skip Ribbon Commands
Skip to main content

Audiovisual Speech Processing

Our lab studies whether and how infants, children, and adults use lip-reading and other visual speech cues to better understand speech in noisy backgrounds.

  • Overview

    Our lab studies whether and how infants, children, and adults use lip-reading and other visual speech cues to better understand speech in noisy backgrounds. We also study how hearing loss and hearing aids affect our ability to use visual speech cues.

    Our studies include:

    • analyzing the physical relationship between acoustic and visual speech signals
    • testing how well adults and children perceive auditory and audiovisual speech in different backgrounds
    • looking for perceptual, cognitive, and linguistic skills that underlie development of the ability to use visual speech cues

      We collaborate with the Audibility, Perception, and Cognition Laboratory and the Human Auditory Development Laboratory, and depend on the Center for Perception and Communication in Children for support in conducting our experiments. If you are interested in participating in our studies, please sign up here to join our list of research volunteers or contact Dr. Kaylah Lalonde or a member of the research team at BTNRH-AVSpeechLab@boystown.org.
  • Current Projects

    When we communicate face-to-face, being able to see the person we're talking to makes it easier to understand speech, especially when the acoustic signal is degraded by noise or hearing loss. We call these benefits audiovisual speech enhancement.

    Development of audiovisual speech enhancement in children
    Visual speech can help us to know when to listen, supplement masked auditory speech information, and help to separate speech from similar competing sounds. Our first project is investigating how well children at various ages can use visual speech in these different ways. Experiments examine how sensitive children are to different audiovisual cues and how much these different mechanisms contribute to individual differences in children's audiovisual speech enhancement. This project's long term goals are to provide a cohesive theoretical account of the development of AV speech enhancement, and ultimately to improve AV communication outcomes for children with hearing loss. This project is funded by a NIH Centers for Biomedical Research Excellence (COGRE) grant (NIH-NIGMS / 5P20GM109023-04).

    Effects of hearing aid compression on temporal cues in audiovisual speech
    Our second project is investigating how wide dynamic range compression (WDRC)—a key feature in hearing aids—affects the timing cues in audiovisual speech, and consequently affect listeners' ability to benefit from visual speech cues. Hearing aids and visual speech (lip reading) are two strategies recommended by audiologists to enhance speech understanding in noise. Mouth movements help listeners to track the temporal amplitude envelope of auditory speech (the slow time-varying changes in signal energy). This help listeners direct auditory analyses to the speech signal of interest, rather than surrounding background noise. Hearing aids with WDRC amplify low intensity parts of auditory signals more than the higher-intensity parts, in order to make more of the signal audible. This distorts the temporal amplitude envelope of auditory speech. [12] Distortions of the auditory amplitude envelope likely disrupt the natural correspondence between mouth movements and the temporal amplitude envelope of auditory speech. Current experiments examine which timing cues are important for audiovisual speech enhancement, how WDRC affects the correspondence between mouth movements and the temporal amplitude envelope of auditory speech, and whether WDRC affects audiovisual speech enhancement in individuals with normal hearing. The long-term goal of this project is to understand the impact of WDRC on audiovisual speech perception in individuals with hearing loss. This project is funded by the Greater Plains IDeA Clinical and Translation Research Grant, through the National Institute of General Medical Sciences, 1U54GM115458-01.

  • Our Team

    Principal Investigator
    Kaylah Lalonde is director of the Audiovisual Speech Processing Lab. Dr. Lalonde completed her Ph.D. in Speech and Hearing Sciences at Indiana University. She completed three years of postdoctoral training on the Auditory Neuroscience Training Grant and in Dr. Lynne Werner's Infant Hearing Lab at the University of Washington. Her postdoctoral research on "The interaction of auditory and visual detection, discrimination, and recognition in infants' real-world environments" was partly funded by an NIH-NIDCD individual postdoctoral fellowship.

    At Boys Town National Research Hospital, Dr. Lalonde directs a project entitled "Development of Audiovisual Speech Enhancement in Children" that is funded through a Centers for Biomedical Research Excellence Grant from the NIH. She is also principal investigator on an NIH-NIGMS funded pilot grant through the Greater Plains IDeA Clinical and Translation Research network, entitled "Effects of hearing aid compression on temporal cues in audiovisual speech."

    Dr. Lalonde's research focuses on the development of speech perception from infancy to adulthood. Her work examines the contribution of auditory, visual, cognitive, linguistic, stimulus and task-related factors to performance on speech perception tasks—and thus to development in general. Her long-term goal is to form a unified account of the development of AV speech perception in noise. To do so, her work focuses on bridging infant and child speech perception literatures and characterizing the development of the mechanisms underlying audiovisual speech enhancement.

    Research Assistant
    Jamie Petersen graduated from the University of Nebraska-Omaha in December 2018, with a bachelor's degree in communication disorders. She is currently applying for graduate programs, to pursue a clinical doctorate of Audiology. Jamie has worked with several other researchers at Boys Town National Research Hospital. 

  • Participate

    For Parents

    When we listen to people speak, there is often noise in the background. This diminishes our ability to perceive speech. As adults, we look at visual cues on the talker's face (i.e., lipreading) to compensate for noisy environments. Children have more difficulty understanding speech in noisy environments, and less is known about their ability to use visual speech cues in noisy. The purpose of this study is to learn how children use lip reading or other visual information on a talker's face to help understand speech in noisy backgrounds. For this research, we test children between 5 and 19 years of age and adults between 19 and 35 years of age.

    Research Participation

    If you or your child decides to participate in our research, here are a few things to know:

    • Tasks may include vision and hearing test, language tests, and tests of how well you/your child can detect and/or recognize speech in noise.
    • You/your child will be provided with frequent breaks filled with fun activities.
    • All of your/your child's information will be kept confidential.
    • Your/your child's time spent with us will be compensated.

      If you are interested in participating in our research or have any questions, please contact us at BTNRH-AVSpeechLab@boystown.org. We would love to hear from you!