Facebook shows off how you’ll use its neural wristbands with AR glasses
Facebook has offered a glimpse inside its plans for a new augmented reality interface, based on technology from CTRL-Labs, the startup it acquired in 2019. In a video, it shows off wristbands that use electromyography (EMG) to translate subtle neural signals into actions — like typing, swiping, or playing games like an archery simulator. The bands also offer haptic feedback, creating a system that’s more responsive than basic hand tracking options.
Facebook Reality Labs published a blog post detailing its work on a prototype of the wristbands. At its simplest, the bands would track basic gestures Facebook calls “clicks,” which are supposed to be reliable and easy to execute. They’re a little bit like the all-purpose Microsoft HoloLens “air tap” gesture but tracked with the nerve signals that run along your arms, rather than visual sensors mounted on a headset.
The bands can theoretically do a lot more, though. For instance, they could track the nerve signals your brain sends to your fingers while you’re typing, so you can type on a virtual keyboard without physical buttons. And unlike a normal keyboard, the bands can slowly adapt to the way you type — so they can “learn” the ways your fingers move when you’re making common typos, then automatically correct for them and capture what you probably meant to type instead.
This would be a huge change in how most people interact with computers, but conceptually, it’s not actually a major update to how CTRL-Labs described its work years ago. In fact, the ultimate possibilities for EMG wristbands are much more mind-bending: eventually, you could perform the same typing-style gestures by thinking about moving your hands instead of actually moving them. Facebook wants to further streamline user interactions by relying on artificial intelligence and augmented reality glasses, which it announced it was working on last year.
Even in their simpler iterations, these controllers would offer an interface that you could wear all the time instead of picking up and holding, like the current Oculus Touch VR controllers. The effect could be similar to smaller startup offerings like the Mudra Band, which senses gestures via an Apple Watch band.
One major new addition is haptics. Facebook says it’s been implementing various prototypes that could give you subtle feedback using different methods. One, the “Bellowband,” has eight pneumatic bellows placed around each wrist. These can be inflated or deflated in patterns that produce distinct sensations. Another is “Tasbi,” which uses vibrating actuators and a “novel wrist squeeze mechanism.” When they’re paired with visual feedback from an AR headset, they can offer a lot of information through a simple and intuitive interface.
Facebook insists that although the band reads neural signals, “this is not akin to mind reading.” Here’s how it explains the concept:
You have many thoughts and you choose to act on only some of them. When that happens, your brain sends signals to your hands and fingers telling them to move in specific ways in order to perform actions like typing and swiping. This is about decoding those signals at the wrist — the actions you’ve already decided to perform — and translating them into digital commands for your device.
CTRL-Labs has still characterized this technology as a brain-computer interface, but it’s a sharp contrast with technologies like Elon Musk’s Neuralink — which reads neural activity directly from the brain through an implant. Implants have unique uses, particularly for people with paralysis or amputated limbs, whose bodies simply can’t send nervous signals to a wristband. But Facebook CEO Mark Zuckerberg recently criticized implants as a near-term consumer technology, saying that “we don’t think that people are going to want to get their head drilled open in order to use virtual or augmented reality.” Wristbands also don’t have quite the same privacy scare factor as something that reads your thoughts at the source.
That said, the bands will almost certainly be collecting a lot of data. That might include incredibly fine variations in typing patterns; overall levels of bodily tension; and any biometric information captured by fitness tracking sensors, augmented reality glasses, and other tech that could be integrated with the bands. (Facebook Reality Labs notes that it has a “neuroethics program” examining the privacy, security, and safety implications of AR and neural interface tech.)
Like most wearable technology, EMG bands offer an intimate look at how our bodies are moving — and while it’s not quite as creepy-sounding as a band that reads your thoughts, it still requires a lot of trust.
Facebook has offered a glimpse inside its plans for a new augmented reality interface, based on technology from CTRL-Labs, the startup it acquired in 2019. In a video, it shows off wristbands that use electromyography (EMG) to translate subtle neural signals into actions — like typing, swiping, or playing games…
