{"id":652502,"date":"2025-02-06T21:10:20","date_gmt":"2025-02-06T18:10:20","guid":{"rendered":"https:\/\/en.buradabiliyorum.com\/improved-brain-decoder-holds-promise-for-communication-in-people-with-aphasia\/"},"modified":"2025-02-06T21:10:20","modified_gmt":"2025-02-06T18:10:20","slug":"improved-brain-decoder-holds-promise-for-communication-in-people-with-aphasia","status":"publish","type":"post","link":"https:\/\/buradabiliyorum.com\/en\/improved-brain-decoder-holds-promise-for-communication-in-people-with-aphasia\/","title":{"rendered":"#Improved brain decoder holds promise for communication in people with aphasia"},"content":{"rendered":"
\n
\n
\n
\n \"Improved
\n A test subject lies in an fMRI scanner at UT Austin listening to podcasts to help train a brain decoder that can translate a person’s thoughts into continuous text. Credit: Nolan Zunk\/University of Texas at Austin.
\n <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n

People with aphasia\u2014a brain disorder affecting about a million people in the U.S.\u2014struggle to turn their thoughts into words and comprehend spoken language.<\/p>\n

A pair of researchers at The University of Texas at Austin have demonstrated an AI-based tool that can translate a person’s thoughts into continuous text, without requiring the person to comprehend spoken words. And the process of training the tool on a person’s own unique patterns of brain activity only takes about an hour.<\/p>\n

This builds on the team’s earlier work creating a brain decoder that required many hours of training on a person’s brain activity as they listened to audio stories. This latest advance suggests it may be possible, with further refinement, for brain computer interfaces to improve communication in people with aphasia.<\/p>\n

“Being able to access semantic representations using both language and vision opens new doors for neurotechnology<\/a>, especially for people who struggle to produce and comprehend language,” said Jerry Tang, postdoctoral researcher at UT in the lab of Alex Huth and first author on a paper describing<\/a> the work in Current Biology<\/i>. “It gives us a way to create language-based brain computer interfaces without requiring any amount of language comprehension.”<\/p>\n

\n
\n
\n
\n
\n
\n
\n
Single-trial BOLD fMRI responses were recorded and decoded while subject S3 watched a self-contained clip from the short film ‘Sintel’ without sound (Blender Foundation; https:\/\/www.sintel.org<\/a>). Each frame is shown at the time it was presented to the subject, and each decoded word is shown at its predicted time. Credit: Current Biology<\/i> (2025). DOI: 10.1016\/j.cub.2025.01.024<\/figcaption><\/figure>\n

In earlier work, the team trained a system, including a transformer model similar to the kind used by ChatGPT, to translate a person’s brain activity into continuous text. The resulting semantic decoder can produce text whether a person is listening to an audio story, thinking about telling a story or watching a silent video that tells a story. But there are limitations.<\/p>\n

To train this brain decoder, participants had to lie motionless in an fMRI scanner for about 16 hours while listening to podcasts, an impractical process for most people and potentially impossible for someone with deficits in comprehending spoken language. And the original brain decoder only works on people for whom it was trained.<\/p>\n

With this latest work, the team has developed a method to adapt the existing brain decoder, trained the hard way, to a new person with only an hour of training in an fMRI scanner while watching short, silent videos, such as Pixar shorts. The researchers developed a converter algorithm that learns how to map the brain activity of a new person onto the brain of someone whose activity was previously used to train the brain decoder, leading to similar decoding at a fraction of the time with the new person.<\/p>\n

\n
\n
\n \"Improved
\n Brain activity from two people watching the same silent film. The UT Austin team developed a converter algorithm that transforms one person’s brain activity (left) into the predicted brain activity of the other person (right), which is a crucial step in adapting their brain decoder to a new subject. Credit: Jerry Tang\/University of Texas at Austin.
\n <\/figcaption><\/figure>\n<\/p><\/div>\n<\/div>\n

Huth said this work reveals something profound about how our brains work: our thoughts transcend language.<\/p>\n

“This points to a deep overlap between what things happ<\/a>en in the brain when you listen to somebody tell you a story, and what things happen in the brain when you watch a video that’s telling a story,” said Huth, associate professor of computer science<\/a> and neuroscience and senior author.<\/p>\n

“Our brain treats both kinds of story as the same. It also tells us that what we’re decoding isn’t actually language. It’s representations of something above the level of language that aren’t tied to the modality of the input.”<\/p>\n

The researchers noted that, just as with their original brain decoder, their improved system works only with cooperative participants who participate willingly in training. If participants on whom the decoder has been trained later put up resistance\u2014for example, by thinking other thoughts\u2014results are unusable. This reduces the potential for misuse.<\/p>\n