Mind-reading AI can translate brainwaves into written text

Using only a sensor-filled helmet combined with artificial intelligence, a team of scientists has announced they can turn a person’s thoughts into written words.

In the study, participants read passages of text while wearing a cap that recorded electrical brain activity through their scalp. These electroencephalogram (EEG) recordings were then converted into text using an AI model called DeWave.

Chin-Teng Lin at the University of Technology Sydney (UTS), Australia, says the technology is non-invasive, relatively inexpensive and easily transportable.

While the system is far from perfect, with an accuracy of approximately 40 per cent, Lin says more recent data currently being peer-reviewed shows an improved accuracy exceeding 60 per cent.

In the study presented at the NeurIPS conference in New Orleans, Louisiana, participants read the sentences aloud, even though the DeWave program doesn’t use spoken words. However, in the team’s latest research, participants read the sentences silently.

Last year, a team led by Jerry Tang at the University of Texas at Austin reported a similar accuracy in converting thoughts to text, but MRI scans were used to interpret brain activity. Using EEG is more practical, as subjects don’t have to lie still inside a scanner.

The DeWave model was trained by looking at lots of examples where brain signals match up with specific sentences, says team member Charles Zhou at UTS.

“For instance, when you think about saying ‘hello’, your brain sends out certain signals,” says Zhou. “DeWave learns how these signals relate to the word ‘hello’ by seeing many examples of these signals for different words or sentences.”

Once DeWave understood the brain signals well, the team connected it to an open-source large language model (LLM), akin to the AI that powers ChatGPT.

“This LLM is like a brainy writer that can make sentences. We tell this writer to pay attention to the signals from DeWave and use them as a guide to create sentences,” says Zhou.

Finally, the team trained both DeWave and the language model together to get even better at writing sentences based on the EEG data.

With further refinement, the researchers predict that the system could revolutionise communication for people who have lost speech, such as those affected by a stroke, and could also have applications in robotics.

Craig Jin at the University of Sydney says he is impressed with the work by Lin’s team. “It’s excellent progress,” he says.

“People have been wanting to turn EEG into text for a long time and the team’s model is showing a remarkable amount of correctness. Several years ago, the conversions from EEG to text were complete and utter nonsense.”