Enlarge /. Facebook is developing a wrist-worn wearable that tracks the nerve activity that controls your hands and fingers. The design could enable new types of human-computer interactions.
It first appeared on March 9 as a tweet on Andrew Bosworth's timeline, the tiny corner of the internet that offers a rare glimpse into the mind of a Facebook manager these days. Bosworth, director of Facebook's augmented and virtual reality research labs, had just published a blog post setting out the company's 10-year vision for the future of human-computer interaction. He then shared a photo of an as-yet-invisible wearable device in a follow-up tweet. Facebook's vision for the future of computer interaction would seem to be to have something that looks like an iPod Mini attached to your wrist.
Facebook already owns our social experience and some of the world's most popular messaging apps – for better or worse. Every time the company delves into hardware it is noticed whether it is a very good VR headset or a video chat device that follows your every move. And it not only triggers intrigue, but also questions: Why does Facebook want to own this new computer paradigm?
In this case, the unanswered questions are less about the hardware itself than about the research behind it – and whether the new interactions Facebook envisions will only deepen our connections with Facebook. (Answer: probably.) In a media briefing earlier this week, Facebook executives and researchers provided an overview of this technology. In the simplest sense, Facebook was testing new computer input with a sensor-filled wearable wrist.
It's an electromyography device that converts electromotive nerve signals into digital commands. If it's your wrist, you can simply slide your fingers around the room to control virtual inputs, whether you're wearing a VR headset or interacting with the real world. You can also “train” it to sense the intent of your fingers so that actions take place even when your hands are completely still.
Enlarge /. Facebook's vision for its wrist-worn device is to be able to type on a virtual desktop keyboard.
This wearable wrist has no name. It's just a concept, and there are different versions of it, some of which have haptic feedback. According to Bosworth, it could take five to ten years for the technology to become widely available.
advertising
All of this has to do with Facebook's plans for virtual and augmented reality. These technologies can result in the user at times experiencing a marked lack of freedom of choice with regard to their hands. When you put on a VR headset, your hands disappear completely. With a pair of hand controls in hand, you can play games or grab virtual objects. However, you then lose the ability to take notes or draw precisely. Some AR or mixed reality headsets like Microsoft's HoloLens have cameras that track spatial gestures. So you can use certain hand signals and the headset will interpret these signals. This works sometimes. So Facebook used this wearable EMG in its virtual reality lab to see if such a device could enable more precise hand-computer interactions.
But Facebook has visions for this wrist technology beyond AR and VR, says Bosworth. "If you really had access to an interface through which you could enter or use a mouse – without having to physically enter or use a mouse, you could use it anywhere." The keyboard is a prime example, he says; This wrist computer is just another means of intentional input unless you can take it with you wherever you go.
Bosworth also suggested the kitchen microwave as a use case – while making it clear that Facebook actually doesn't build a microwave. The interfaces for household appliances are all different. So why not program such a device so that it simply understands when you want to cook something for 10 minutes on medium power?
The virtual demo that Facebook gave earlier this week showed a gamer wearing the wrist device and controlling a character in a rudimentary video game on a flat screen without moving his fingers. These types of demos tend to be forgiving of the pun (pardon the pun) which Bosworth insisted it wasn't. In this case, he said, the mind generates signals identical to those that would move the thumb, but the thumb does not move. The device records an explicit intention to move the thumb. "We don't know what's going on in the brain, which is full of thoughts, ideas, and imaginations. We don't know what is happening until someone sends a signal over the cable."
Bosworth also pointed out that this wearable wrist is different from the invasive implants used in a 2019 study of brain-computer interfaces that Facebook collaborated with the University of California at San Francisco. and unlike Elon Musk's Neonink, a wireless implant that theoretically allows people to send neuroelectric signals from their brains directly to digital devices. In other words, Facebook doesn't read our minds even if it already knows a lot about what's going on in our heads.
advertising
Researchers say there is still a lot of work to be done in the area of using EMG sensors as virtual input devices. Precision is a big challenge. Chris Harrison, the director of the Future Interfaces Group at Carnegie Mellon University's Human-Computer Interaction Lab, points out that every person's nerves are slightly different, as are the shapes of our arms and wrists. “There is always a calibration process that must be performed with any muscle sensor system or BCI system. It really depends on where the computer intelligence is, ”says Harrison.
Enlarge /. A closer look at the portable prototype.
And even with haptic feedback integrated into these devices, as Facebook does with some of its prototypes, there is a risk of visual-haptic mismatches in which the visual experience of the user – whether in AR, VR or in real space – does not correlate with the haptic one Reaction. These points of friction can make these human-computer interactions feel frustratingly unreal.
Even if Facebook can overcome these barriers in its research labs, the question still arises as to why Facebook – largely a software company – would want to own this new computing paradigm. And should we trust him? This extremely powerful tech company that has been shown to have exchanged user data in exchange for other things of equal or more value, as WIRED's Fred Vogelstein wrote in 2018? A recent report in MIT Technology Review shows how a team gathered on Facebook to crack down on "responsible AI" and was undermined by leadership's relentless pursuit of growth.
Facebook executives said this week that these new human-computer interaction devices will run as many computers as possible "on the device," meaning the information will not be leaked to the cloud. However, Bosworth will not commit to how much data will ultimately be shared with Facebook or how that data will be used. The whole thing is a prototype, so there is still nothing essential to pull apart, he says.
"Sometimes these companies have stacks of cash that are big enough to basically invest in these huge R&D projects and they will take a loss on things like that if that means they can be frontrunners in the future," says Michelle Richardson, director of the data and privacy project at the nonprofit Center for Democracy and Technology. "But with companies of all sizes and products, once built, it's so difficult to overtake." Anything that can start the conversation about before the devices are built is a good thing. "
According to Bosworth, Facebook wants to lead this next paradigm shift in computing as the company sees such technologies as fundamental to connecting people. If anything, the past year has shown us how important it is to socialize – to feel personal, says Bosworth. He also seems to believe that he can earn the trust he needs by not “surprising” customers. "You say what you do, you set expectations, and you meet those expectations over time," he says. "Trust comes on foot and goes on horseback." Pink AR glasses, activated.
This story originally appeared on wired.com.
