For years, Pancho communicated by spelling words on a computer with a pointer attached to a baseball cap, a tedious method that allowed him to type about five correct words per minute.
“I had to bend / lean my head forward and put one key letter in a row to write it,” he wrote via email.
Last year, the researchers gave him another device with a head-controlled mouse, but it’s still not nearly as fast as the brain electrodes in the research sessions.
Pancho communicated 15 to 18 words per minute through the electrodes. That was the maximum rate the study allowed because the computer waited between prompts. Dr. Chang says faster decoding is possible, although it’s unclear whether it will reach the pace of a typical conversation: around 150 words per minute. Speed is a major reason the project focuses on speaking by typing directly into the brain’s word production system, rather than hand movements while typing or writing.
“It’s the most natural way to communicate,” he said.
Pancho’s lively personality helped researchers overcome challenges, but it also occasionally makes speech recognition uneven.
“Sometimes I can’t control my emotions and I laugh a lot and don’t do the experiment too well,” he wrote by email.
Dr. Chang recalled times when, after the algorithm successfully identified a sentence, “he was visibly shaking and it looked like he was giggling.” When that happened, or when he yawned or became distracted during repetitive tasks, “did.” it doesn’t work very well because he wasn’t really focused on understanding those words. So we have to work on some things because of course we want it to work all the time. “
The algorithm sometimes confused words with similar phonetic sounds and identified “go” as “bring”, “do” as “you” and words beginning with “F” – “believe”, “family”, “feel” – as a V word “very”.