UW gesture technology increases efficiency of hands-free use

AllSee b

AllSee b -

Bryce Kellogg, a PhD candidate in the electrical engineering department, demonstrates the gesture technlogy developed by the team.They hope to get the technology in smart phones.

Photo by Anastasia Stepankowsky

AllSee a

AllSee a

In the brief history of gesture-recognition technology, its application has been limited by the large amount of power that it requires. But now AllSee, a gesture-recognition system being developed by UW researchers, is bringing this technology to smartphones and similar devices.

AllSee uses existing signals put out by other electronic devices to read the users’ hand movements, using 1,000 to 10,000 times less power than similar systems. The technology can even operate on devices without batteries.

“Current gesture recognition is limited to high-end power-consuming devices,” said Shyam Gollakota, an assistant professor in the department of computer science and engineering. “AllSee is the first gesture recognition system that can work on devices that harvest energy from TV signals around us.”

In demonstrations, the hardware, which resembles a small circuit attached to a straight piece of wire, was hooked up to a Galaxy Nexus phone and placed into the pocket of the user. Changes in the wireless signals reflected off the user are received and interpreted by the AllSee device. By using a combination of simple finger and hand movements, the user can perform eight different gestures, which AllSee can pick up with an average accuracy of more than 94 percent for TV transmissions.

Gollakota worked on another project, called WiSee, before working with his assistants Bryce Kellogg and Vamsi Talla on AllSee. Kellogg, a graduate student in electrical engineering, explains that the work Gollakota did with WiSee made numerous contributions to AllSee.

“WiSee is the precursor to AllSee in that it uses Wi-Fi signals, but it also uses these Doppler techniques and lots of heavy computation,” said Kellogg. “The idea was, ‘Can we do pretty much everything you can do with WiSee but on a low powered device?’”

A Wi-Fi router equipped with WiSee technology will calculate the changes in distance of the user’s hand based on the signals produced by other devices bouncing off of the user.

AllSee has avoided draining unnecessary power by not using heavy computation. By harnessing the signals put out by other devices, AllSee avoids using precious energy to create its own signal.

Kellogg became the source for most of the code written for the project. According to Kellogg, the project came about through collaboration with the UW Sensor Systems Laboratory, a part of the Department of Computer Science and Engineering.

Talla, also a graduate student in electrical engineering, designed the hardware used in the project. Talla became involved after working with Gollakota on the backscatter project, a project also conducted at UW that uses a small transmitter that doesn’t require electricity to send signals. The transmitter picks up on other signals around it, using those signals to send messages. The same technology used in AllSee.

“The radio frequency front on the two projects are similar in that they harvest power from ambient signals,” Talla said.

Gollakota says that he is already looking toward the future of the technology. Due to the small amounts of power that the device uses, he sees no limit to its applications.

“We are talking to industry folks about integrating it with mobile phones to enable gesture recognition on your phone even when your phone is in your pocket or in a bag,”  said Gollakota.

AllSee could also make it easier and more energy-efficient to communicate with home appliances such as alarm systems. The researchers are also starting to look further ahead, to a future where AllSee has a place in what those in the field call “the Internet of things,” where physical objects can be interacted with, and can interact with each other, through wired and wireless networks.

“We would be gesturing with objects, because really it’s a more natural way to interact with things,” Kellogg said.

Reach contributing writer Matt Spaw at development@dailyuw.com. Twitter: @Matt_Spaw

Please read our Comment policy.