In everyday life, we automatically and effortlessly decode speech into language independently of who speaks. Similarly, we recognize a speaker's voice independently of what he says. Formisano et al from a recent Science paper show that it is possible to decode the contents of speech and the identity of the speaker from measurements of the brain activity of a listener. They map and decode, trial-by-trial, the spatially distributed activation patterns evoked by listening to different vowels or speakers evoke in distinct patches of the listeners' auditory cortex. The pattern associated with a vowel does not change if the vowel is spoken by another speaker and the pattern associated with a speaker does not depend on what the person says:
"Who" Is Saying "What"? Brain-Based Decoding of Human Voice and Speech
Can we decipher speech content ("what" is being said) and speaker identity ("who" is saying it) from observations of brain activity of a listener? Here, we combine functional magnetic resonance imaging with a data-mining algorithm and retrieve what and whom a person is listening to from the neural fingerprints that speech and voice signals elicit in the listener's auditory cortex. These cortical fingerprints are spatially distributed and insensitive to acoustic variations of the input so as to permit the brain-based recognition of learned speech from unknown speakers and of learned voices from previously unheard utterances. Our findings unravel the detailed cortical layout and computational properties of the neural populations at the basis of human speech recognition and speaker identification.
Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, University of Maastricht, 6200 MD Maastricht, Netherlands. E-mail: e.formisano@psychology.unimaas.nl
