WiFi gesture recognition lets you control objects through walls

Imagine, with the wave of a hand, adjusting the thermostat without getting out of bed, or turning up the music in the other room while in the shower. WiSee, a new gesture-recognition system, aims to harness the ever-present wireless Internet signals blanketing people’s homes to allow remote control of all their electronics.

Since walls pose no obstacle for WiFi’s radio waves to traverse, such a system could work throughout the entire house, even if users are several rooms removed from the appliance they’re controlling.

WiFi gesture recognition lets you control objects through walls

“This is something people have been thinking about for some time,” says Shyamnath Gollakota, a University of Washington computer scientist. “But we asked: Can we scale it to much larger spaces, say in the whole home or building?”

To provide full-home coverage, other systems currently or soon to be on the market like Microsoft’s Kinect or Thalmic’s MYO, would require either rigging up cameras throughout the house or donning wearable electronics. Such technologies can raise privacy questions or be cumbersome, Gollakota argues. Indeed, he first thought of using WiFi to achieve camera- and gadget-free gesture control while playing with his brother’s Xbox Kinect and becoming frustrated because he kept falling out of the device’s motion-sensing area.

Watching the Waves

Inspired to find a different approach to gesture control, around nine months ago Gollakota and his colleagues began building WiSee. To create their system, they took advantage of a straightforward phenomenon—the Doppler effect, or the way a wave’s frequency changes when the observer or wave’s source move with respect to each other. The most well known example, taught in high school science classes around the world, is the way a train’s whistle pitch sounds higher when it’s approaching a stationary observer and lower after it passes and continues down the tracks.

Waving a hand or kicking a foot, the researchers found, creates minute frequency shifts of about 10 hertz in wireless signals, which have a bandwidth of around 20 megahertz—a difference of six orders of magnitude. Gollakota and his team wrote algorithms for detecting the tiny shift that such movements cause in the WiFi’s bandwidth.

So far, their system, which will premiere at a mobile computing and networking conference in the fall, recognizes nine different gestures, including kicking, punching and various forms of hand waving. In the research team’s paper, they report up to 94% accuracy in the WiSee’s ability to recognize these motions in tests carried out with five users performing 900 gestures in a two-bedroom apartment and an office. They think that standard WiFi routers could eventually be tweaked to enable WiSee functionality.

Hurdles Still to Surmount

Multiple antennas allow WiSee to tune into a specific person’s movements in the house, meaning that up to five people could potentially use a single WiSee device without causing confusion. “In a typical home, there are multiple people, and each person would have their own signature,” Gollakota says.

When four people were involved simultaneously in the trials, however, WiSee’s accuracy dropped to around 60%, so this functionality still needs to be improved.

WiSee will eventually take into account where in the home a user is, providing location-related context for translating a certain command into a given action for a specific device. “A user in the kitchen can perform a motion to turn on the oven, while if that user is in the bedroom that motion would correspond to turning up the thermostat,” he says.

WiSee does raise security questions. Without some sort of password system, anyone could walk into a WiSee-equipped home or office and take control of the system. The team is attempting to devise a series of patterned movements that act as a password to allow users to access the system.

In addition to addressing security concerns, they are also testing the limits of gesture recognition, exploring whether the device could be scaled up to recognize 50 different motions, for example, or be programmed to perform safety functions for certain users, like automatically dialing 911 if a homeowner falls down.

Though Gollakota and his colleagues set out primarily to answer a scientific question and have only provided proof of concept with this initial study, the response to their work has been significant, prompting the team to begin speaking with potential collaborators about creating a company based on the WiSee. “The video has more than a quarter million views, and companies — including the big ones — have been contacting us,” he says. “It seems like there is a huge potential.”

This article comes courtesy of the Txchnologist, GE's blog about innovation in science and technology.