When Ann suffered a stroke over 18 years ago, she was left badly paralysed and unable to speak or type. The only way she could communicate was through movement-tracking technology which she used to individually select letters. She could only produce 14 words a minute.
But a remarkable breakthrough has enabled her to speak through a digital avatar that also shows her facial expressions, The Guardian reported. The 47-year-old Ann (only her first name is given) suffered what is known as a brainstem stroke nearly 20 years ago.
Meta’s got nothing on this
“Our goal is to restore a full, embodied way of communicating, which is really the most natural way for us to talk with others,” Prof Edward Chang of the University of California, San Francisco (UCSF), told the paper. “These advancements bring us much closer to making this a real solution for patients.”
Led by Chang, the UCSF team implanted a paper-thin rectangle of 253 electrodes onto the surface of Ann’s brain over a region critical for speech. “The electrodes intercepted the brain signals that, if not for the stroke, would have controlled muscles in her tongue, jaw, larynx and face,” The Guardian wrote.
Ann spent months training an “AI algorithm to detect her unique brain signals for various speech sounds by repeating different phrases repeatedly”.
Read More: How ChatGPT might be able to help the world’s poorest and the organisations that work with them
Having learnt 39 distinctive sounds, the team’s system then used a large language model (LLM) similar to that used by ChatGPT to “translate the signals into intelligible sentences. This was then used to control an avatar with a voice personalised to sound like Ann’s voice before the injury, based on a recording of her speaking at her wedding”.
Dr David Moses, an assistant professor in neurological surgery at UCSF and co-author of the research, said: “Giving people the ability to freely control their own computers and phones with this technology would have profound effects on their independence and social interactions”.
“This is quite a jump from previous results. We’re at a tipping point,” says Prof Nick Ramsey, a University of Utrecht neuroscientist, who was not involved in the research, speaking with The Guardian.