Facebook is working with universities to develop mind-reading tech

Two years ago, Facebook said it wanted to create a headband using mind-reading technology whereby people would be able to type by simply imagining the words. 

Current advancements in brain-computer interface (BCI) technologies dictate that this is only possible through the use of implanted electrodes. Facebook’s dream is to do this in a non-invasive manner, such as using wearables like a headband. 

According to Mark Chevillet, a Research Director at Facebook Reality Labs (FRL), such a device wouldn’t be only relevant for those who cannot speak, because why stop there when it could also be used to help people interact with their digital devices. 

“Most people have used the voice assistant on their phone, but they hardly ever use it in front of other people,” he points out.

As Chevillet notes, voice-assisted technologies are great for the connected home but isn’t always practical everywhere else. When the user enters a crowded space, the technology is no longer useful. 

Mind-reading could be the gamechanger.

Recognising its potential, Facebook has since been pouring funding into it. And in a recent update on its website, social media giant revealed it is a step closer to achieving that dream.

The research

Facebook’s research partners at the University of California, San Francisco, have been able to decode spoken words and phrases by simply analysing the brain signals of its human volunteers using implanted electrodes. 

In the experiment, the subjects listened to multiple-choice questions and spoke the answers out loud. The electrode recorded activities in the brain related to understanding and producing speech, and looked for patterns to match the specific words and phrases in real-time.

The results were between 61 to 76 percent accurate.

It had previously taken the researchers weeks or months to translate brain activity into speech via offline methods. This isn’t ideal for the intended objective of the UCSF study, ie. to restore voices to those who have lost their ability to speak. 

But now being able to do this in real-time and with a high level of accuracy changes everything. The breakthrough indicates the world may not be that far away from developing Facebook’s mind-reading headband.

Imagined Input for #AR Glasses: BCI could offer hands-free communication without saying a word. Check out the latest entry in our Inside @facebook Reality Labs blog series to learn more // https://t.co/vF9YQR6hx9 pic.twitter.com/VcwhDtogJL

— TechAtFacebook (@techatfacebook) July 30, 2019

Of course, UCSF researchers say their algorithm is only able to reconise a small set of words and phrases. Their ultimate goal is to achieve a real-time decoding speed of 100 words per minute, with a 1,000-word vocabulary and a word-error rate of less than 17 percent.

The work published in a paper in Nature Communications involved volunteer research participants with normal speech who were undergoing brain surgery for epilepsy. 

According to Facebook, it is part of a larger research programme at UCSF it provides support for via a project it calls Project Steno.

The firm doesn’t offer further detail on the arrangement with UCSF or any of its other university partners, just that it is providing funding and research support. A small team of Facebook researchers are working directly with Eddie Chang, the world-renowned neurosurgeon who leads the UCSF team, to provide input and engineering support, the firm says.

According to MIT Technology Review, a UCSF spokesman declined to provide a copy of the research contract or consent forms signed by research volunteers. The spokesman, Nicholas Weiler, reportedly confirmed the consent forms list Facebook among several potential sponsors of the research.

In addition to UCSF, Facebook is also exploring the possibilities of non-invasive BCI methods with other higher education institutions, namely the Mallinckrodt Institute of Radiology at the Washington University School of Medicine and the Applied Physics Laboratory (APL) at John Hopkins University.

Chevillet who leads Facebook’s mind-reading project, worked at APL prior to joining the FRL. According to Facebook, it was Chevillet who connected with Chang to advance Facebook’s work in BCI. 

Before building any kind of wearable, FRL first needed to know whether a silent speech interface was even possible. At UCSF, Chang leads a research team dedicated to developing treatments for patients with neurological disorders.

As it turned out, Chang had long wanted to develop a communication device for patients who had lost their ability to speak due to a brain injury. The match, therefore, made perfect sense.

Privacy concerns

For the rest of the world, their research is important for a variety of reasons. The obvious one is the technology’s potential to return speech to those who have lost the ability.

Second to that would be its commercial use cases, applicable to any activity that involves humans communicating with machines.

But Facebook’s involvement, and increasing interest among big tech firms to jump the mind-reading bandwagon, raises privacy and ethical concerns. Earlier this month, Neuralink, a brain interface firm formed by SpaceX’s Elon Musk, said it hoped to implant electrodes into the brains of paralysed volunteers within two years to help them control their digital devices.

What this means is that these firms would be able to get a hold of data… right from the brains of its users. What they do with that data, is the biggest question, especially when regulation is often so slow to keep pace with new technologies. Responsible innovation is a major concern that countries everywhere are grappling with as technological progress continues at breakneck speeds.

Facebook, however, insists that for the UCSF research, de-identified patient data collected at the university remains at the university.

“Facebook researchers have limited access to de-identified data, which remain onsite at UCSF and under its control at all times,” the company says. 

It adds that while it has some ways to go before achieving what it wants, it’s not too soon for conversations to be had on the potential of mind-reading technology, especially with regards to privacy concerns.

“Just because we aren’t all trained bioethicists, that doesn’t mean we can’t or shouldn’t be part of the conversation,” says Emily Mugler, an engineer on the BCI team at FRL.

“In fact, it’s our responsibility to make sure everyone is made properly aware of the scientific advances, so we can all have an informed discussion about the future of this technology.”

It’s for this reason, Facebook says, that it has openly spoken about, supported and encouraged its partner institutions to publish their research in journals.